[==========] Running 56 tests from 4 test suites.
[----------] Global test environment set-up.
[----------] 30 tests from SecurityITest
[ RUN ] SecurityITest.TestAuthorizationOnListTablets
Loading random data
Initializing database '/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786517687384214-19665-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 06:54:47 dist-test-slave-65mx krb5kdc[19674](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 06:54:47 dist-test-slave-65mx krb5kdc[19674](info): set up 2 sockets
Aug 12 06:54:47 dist-test-slave-65mx krb5kdc[19674](info): commencing operation
krb5kdc: starting...
WARNING: Logging before InitGoogleLogging() is written to STDERR
W20260812 06:54:48.659637 19665 mini_kdc.cc:121] Time spent starting KDC: real 1.198s user 0.006s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786517687384214-19665-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786517687384214-19665-0/krb5kdc/test-user.keytab.
Aug 12 06:54:48 dist-test-slave-65mx krb5kdc[19674](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517688, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T06:54:48Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T06:54:48Z Disabled control of system clock
WARNING: no policy specified for kudu/127.19.52.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.52.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.52.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786517687384214-19665-0/krb5kdc/kudu_127.19.52.126.keytab.
Entry for principal kudu/127.19.52.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786517687384214-19665-0/krb5kdc/kudu_127.19.52.126.keytab.
I20260812 06:54:48.808512 19665 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu
/tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786517687384214-19665-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786517687384214-19665-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786517687384214-19665-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786517687384214-19665-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.52.126:42493
--webserver_interface=127.19.52.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786517687384214-19665-0/krb5kdc/kudu_127.19.52.126.keytab
--principal=kudu/127.19.52.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.52.84:38137
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.19.52.126:42493
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786517687384214-19665-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786517687384214-19665-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786517687384214-19665-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 06:54:48.939997 19688 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:54:48.940315 19688 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:54:48.940361 19688 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 06:54:48.944681 19688 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 06:54:48.944761 19688 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 06:54:48.944782 19688 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 06:54:48.944799 19688 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 06:54:48.944815 19688 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 06:54:48.952514 19688 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.52.84:38137
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786517687384214-19665-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786517687384214-19665-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.19.52.126:42493
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786517687384214-19665-0/krb5kdc/kudu_127.19.52.126.keytab
--principal=kudu/127.19.52.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.52.126:42493
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786517687384214-19665-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.19.52.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19688
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786517687384214-19665-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 06:15:54 UTC on ccycloud.quasar-rnfzce.root.comops.site
build id 4581989
I20260812 06:54:48.954155 19688 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:54:48.955770 19688 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260812 06:54:48.969556 19688 server_base.cc:1034] running on GCE node
W20260812 06:54:48.969599 19696 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:54:48.969901 19695 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:54:48.971180 19699 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:54:48.971967 19688 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:54:48.973930 19688 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:54:48.975246 19688 hybrid_clock.cc:648] HybridClock initialized: now 1786517688975184 us; error 108 us; skew 500 ppm
Aug 12 06:54:48 dist-test-slave-65mx krb5kdc[19674](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517688, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.52.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 06:54:48.978691 19688 init.cc:377] Logged in from keytab as kudu/127.19.52.126@KRBTEST.COM (short username kudu)
I20260812 06:54:48.980018 19688 webserver.cc:460] Webserver started at http://127.19.52.126:39843/ using document root <none> and password file <none>
I20260812 06:54:48.981069 19688 fs_manager.cc:359] Metadata directory not provided
I20260812 06:54:48.981179 19688 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:54:48.981742 19688 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 06:54:48.984843 19688 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786517687384214-19665-0/minicluster-data/master-0/data/instance:
uuid: "697bf114681a41559e6435682ef23604"
format_stamp: "Formatted at 2026-08-12 06:54:48 on dist-test-slave-65mx"
server_key: "29fe1e9c7fc0384340e55134d5476ba0"
server_key_iv: "f26c9c75f056f75e9a07d9bcb478a2f4"
server_key_version: "encryptionkey@0"
I20260812 06:54:48.985548 19688 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786517687384214-19665-0/minicluster-data/master-0/wal/instance:
uuid: "697bf114681a41559e6435682ef23604"
format_stamp: "Formatted at 2026-08-12 06:54:48 on dist-test-slave-65mx"
server_key: "29fe1e9c7fc0384340e55134d5476ba0"
server_key_iv: "f26c9c75f056f75e9a07d9bcb478a2f4"
server_key_version: "encryptionkey@0"
I20260812 06:54:48.991097 19688 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260812 06:54:48.995213 19707 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 06:54:48.997290 19688 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.000s
I20260812 06:54:48.997534 19688 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786517687384214-19665-0/minicluster-data/master-0/data,/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786517687384214-19665-0/minicluster-data/master-0/wal
uuid: "697bf114681a41559e6435682ef23604"
format_stamp: "Formatted at 2026-08-12 06:54:48 on dist-test-slave-65mx"
server_key: "29fe1e9c7fc0384340e55134d5476ba0"
server_key_iv: "f26c9c75f056f75e9a07d9bcb478a2f4"
server_key_version: "encryptionkey@0"
I20260812 06:54:48.997705 19688 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786517687384214-19665-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786517687384214-19665-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786517687384214-19665-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:54:49.018329 19688 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 06:54:49.022372 19688 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 06:54:49.022691 19688 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:54:49.039779 19688 rpc_server.cc:307] RPC server started. Bound to: 127.19.52.126:42493
I20260812 06:54:49.039871 19768 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.52.126:42493 every 8 connection(s)
I20260812 06:54:49.041375 19688 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786517687384214-19665-0/minicluster-data/master-0/data/info.pb
I20260812 06:54:49.047271 19769 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 06:54:49.048758 19665 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu as pid 19688
I20260812 06:54:49.048898 19665 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786517687384214-19665-0/minicluster-data/master-0/wal/instance
I20260812 06:54:49.050511 19665 external_mini_cluster.cc:1428] Setting key 03d434b655ea12696acf7b1eff6d418a
I20260812 06:54:49.058259 19769 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 697bf114681a41559e6435682ef23604: Bootstrap starting.
I20260812 06:54:49.062981 19769 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 697bf114681a41559e6435682ef23604: Neither blocks nor log segments found. Creating new log.
I20260812 06:54:49.064823 19769 log.cc:824] T 00000000000000000000000000000000 P 697bf114681a41559e6435682ef23604: Log is configured to *not* fsync() on all Append() calls
I20260812 06:54:49.068354 19769 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 697bf114681a41559e6435682ef23604: No bootstrap required, opened a new log
Aug 12 06:54:49 dist-test-slave-65mx krb5kdc[19674](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517688, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.19.52.126@KRBTEST.COM
I20260812 06:54:49.074452 19769 raft_consensus.cc:348] T 00000000000000000000000000000000 P 697bf114681a41559e6435682ef23604 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "697bf114681a41559e6435682ef23604" member_type: VOTER last_known_addr { host: "127.19.52.126" port: 42493 } }
I20260812 06:54:49.074791 19769 raft_consensus.cc:374] T 00000000000000000000000000000000 P 697bf114681a41559e6435682ef23604 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 06:54:49.074899 19769 raft_consensus.cc:729] T 00000000000000000000000000000000 P 697bf114681a41559e6435682ef23604 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 697bf114681a41559e6435682ef23604, State: Initialized, Role: FOLLOWER
I20260812 06:54:49.075646 19769 consensus_queue.cc:260] T 00000000000000000000000000000000 P 697bf114681a41559e6435682ef23604 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "697bf114681a41559e6435682ef23604" member_type: VOTER last_known_addr { host: "127.19.52.126" port: 42493 } }
I20260812 06:54:49.075891 19769 raft_consensus.cc:388] T 00000000000000000000000000000000 P 697bf114681a41559e6435682ef23604 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 06:54:49.076027 19769 raft_consensus.cc:482] T 00000000000000000000000000000000 P 697bf114681a41559e6435682ef23604 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 06:54:49.076172 19769 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 697bf114681a41559e6435682ef23604 [term 0 FOLLOWER]: Advancing to term 1
I20260812 06:54:49.077652 19769 raft_consensus.cc:504] T 00000000000000000000000000000000 P 697bf114681a41559e6435682ef23604 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "697bf114681a41559e6435682ef23604" member_type: VOTER last_known_addr { host: "127.19.52.126" port: 42493 } }
I20260812 06:54:49.078279 19769 leader_election.cc:302] T 00000000000000000000000000000000 P 697bf114681a41559e6435682ef23604 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 697bf114681a41559e6435682ef23604; no voters:
I20260812 06:54:49.078730 19769 leader_election.cc:288] T 00000000000000000000000000000000 P 697bf114681a41559e6435682ef23604 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 06:54:49.078939 19774 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 697bf114681a41559e6435682ef23604 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 06:54:49.079327 19774 raft_consensus.cc:686] T 00000000000000000000000000000000 P 697bf114681a41559e6435682ef23604 [term 1 LEADER]: Becoming Leader. State: Replica: 697bf114681a41559e6435682ef23604, State: Running, Role: LEADER
I20260812 06:54:49.079864 19774 consensus_queue.cc:237] T 00000000000000000000000000000000 P 697bf114681a41559e6435682ef23604 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "697bf114681a41559e6435682ef23604" member_type: VOTER last_known_addr { host: "127.19.52.126" port: 42493 } }
I20260812 06:54:49.080855 19769 sys_catalog.cc:564] T 00000000000000000000000000000000 P 697bf114681a41559e6435682ef23604 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 06:54:49.083384 19775 sys_catalog.cc:455] T 00000000000000000000000000000000 P 697bf114681a41559e6435682ef23604 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "697bf114681a41559e6435682ef23604" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "697bf114681a41559e6435682ef23604" member_type: VOTER last_known_addr { host: "127.19.52.126" port: 42493 } } }
I20260812 06:54:49.083582 19775 sys_catalog.cc:458] T 00000000000000000000000000000000 P 697bf114681a41559e6435682ef23604 [sys.catalog]: This master's current role is: LEADER
I20260812 06:54:49.084275 19771 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:54:49.055528 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36016 (local address 127.19.52.126:42493)
0812 06:54:49.056201 (+ 673us) server_negotiation.cc:206] Beginning negotiation
0812 06:54:49.056288 (+ 87us) server_negotiation.cc:399] Waiting for connection header
0812 06:54:49.057407 (+ 1119us) server_negotiation.cc:407] Connection header received
0812 06:54:49.062017 (+ 4610us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:54:49.062036 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:54:49.062432 (+ 396us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:54:49.062796 (+ 364us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:54:49.065957 (+ 3161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:54:49.067604 (+ 1647us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:54:49.069431 (+ 1827us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:54:49.070228 (+ 797us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:54:49.070479 (+ 251us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:54:49.075035 (+ 4556us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:54:49.075078 (+ 43us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:54:49.075091 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:54:49.075123 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:54:49.077956 (+ 2833us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:54:49.078660 (+ 704us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:54:49.078665 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:54:49.078669 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:54:49.078874 (+ 205us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:54:49.079413 (+ 539us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:54:49.079417 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:54:49.079418 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:54:49.080101 (+ 683us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:54:49.080404 (+ 303us) server_negotiation.cc:1035] Waiting for connection context
0812 06:54:49.081590 (+ 1186us) server_negotiation.cc:299] Negotiation successful
0812 06:54:49.082034 (+ 444us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":458,"thread_start_us":238,"threads_started":1}
I20260812 06:54:49.084352 19784 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 06:54:49.084290 19776 sys_catalog.cc:455] T 00000000000000000000000000000000 P 697bf114681a41559e6435682ef23604 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 697bf114681a41559e6435682ef23604. Latest consensus state: current_term: 1 leader_uuid: "697bf114681a41559e6435682ef23604" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "697bf114681a41559e6435682ef23604" member_type: VOTER last_known_addr { host: "127.19.52.126" port: 42493 } } }
I20260812 06:54:49.087184 19776 sys_catalog.cc:458] T 00000000000000000000000000000000 P 697bf114681a41559e6435682ef23604 [sys.catalog]: This master's current role is: LEADER
I20260812 06:54:49.089609 19784 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 06:54:49.100512 19784 catalog_manager.cc:1347] Generated new cluster ID: 5a698882749e486284e1dfc8c32faff4
I20260812 06:54:49.100664 19784 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 06:54:49.109215 19784 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 06:54:49.110320 19784 catalog_manager.cc:1504] Loading token signing keys...
I20260812 06:54:49.116947 19784 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 697bf114681a41559e6435682ef23604: Generated new TSK 0
I20260812 06:54:49.117775 19784 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.19.52.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.52.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.52.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786517687384214-19665-0/krb5kdc/kudu_127.19.52.65.keytab.
Entry for principal kudu/127.19.52.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786517687384214-19665-0/krb5kdc/kudu_127.19.52.65.keytab.
I20260812 06:54:49.161079 19665 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu
/tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786517687384214-19665-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786517687384214-19665-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786517687384214-19665-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786517687384214-19665-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.52.65:0
--local_ip_for_outbound_sockets=127.19.52.65
--webserver_interface=127.19.52.65
--webserver_port=0
--tserver_master_addrs=127.19.52.126:42493
--keytab_file=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786517687384214-19665-0/krb5kdc/kudu_127.19.52.65.keytab
--principal=kudu/127.19.52.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.52.84:38137
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786517687384214-19665-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786517687384214-19665-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786517687384214-19665-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 06:54:49.331573 19798 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:54:49.331897 19798 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:54:49.331945 19798 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 06:54:49.336524 19798 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 06:54:49.336629 19798 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 06:54:49.336695 19798 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.52.65
I20260812 06:54:49.342667 19798 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.52.84:38137
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786517687384214-19665-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786517687384214-19665-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786517687384214-19665-0/krb5kdc/kudu_127.19.52.65.keytab
--principal=kudu/127.19.52.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.52.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786517687384214-19665-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.19.52.65
--webserver_port=0
--tserver_master_addrs=127.19.52.126:42493
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19798
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.52.65
--log_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786517687384214-19665-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 06:15:54 UTC on ccycloud.quasar-rnfzce.root.comops.site
build id 4581989
I20260812 06:54:49.343811 19798 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:54:49.344851 19798 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 06:54:49.356479 19806 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:54:49.356508 19803 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:54:49.356680 19798 server_base.cc:1034] running on GCE node
W20260812 06:54:49.356890 19804 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:54:49.357434 19798 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:54:49.358165 19798 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:54:49.359335 19798 hybrid_clock.cc:648] HybridClock initialized: now 1786517689359318 us; error 33 us; skew 500 ppm
Aug 12 06:54:49 dist-test-slave-65mx krb5kdc[19674](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517689, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.52.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 06:54:49.363515 19798 init.cc:377] Logged in from keytab as kudu/127.19.52.65@KRBTEST.COM (short username kudu)
I20260812 06:54:49.364682 19798 webserver.cc:460] Webserver started at http://127.19.52.65:46523/ using document root <none> and password file <none>
I20260812 06:54:49.365404 19798 fs_manager.cc:359] Metadata directory not provided
I20260812 06:54:49.365512 19798 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:54:49.365780 19798 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 06:54:49.368363 19798 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786517687384214-19665-0/minicluster-data/ts-0/data/instance:
uuid: "0e4f9bf6f286465aae2354003ada6727"
format_stamp: "Formatted at 2026-08-12 06:54:49 on dist-test-slave-65mx"
server_key: "8cc2295743da9755cb9c66d45cb00618"
server_key_iv: "c12709d3303be4dffc019dea4e5ce85b"
server_key_version: "encryptionkey@0"
I20260812 06:54:49.369174 19798 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786517687384214-19665-0/minicluster-data/ts-0/wal/instance:
uuid: "0e4f9bf6f286465aae2354003ada6727"
format_stamp: "Formatted at 2026-08-12 06:54:49 on dist-test-slave-65mx"
server_key: "8cc2295743da9755cb9c66d45cb00618"
server_key_iv: "c12709d3303be4dffc019dea4e5ce85b"
server_key_version: "encryptionkey@0"
I20260812 06:54:49.373929 19798 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260812 06:54:49.376830 19813 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 06:54:49.378465 19798 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260812 06:54:49.378661 19798 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786517687384214-19665-0/minicluster-data/ts-0/data,/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786517687384214-19665-0/minicluster-data/ts-0/wal
uuid: "0e4f9bf6f286465aae2354003ada6727"
format_stamp: "Formatted at 2026-08-12 06:54:49 on dist-test-slave-65mx"
server_key: "8cc2295743da9755cb9c66d45cb00618"
server_key_iv: "c12709d3303be4dffc019dea4e5ce85b"
server_key_version: "encryptionkey@0"
I20260812 06:54:49.378805 19798 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786517687384214-19665-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786517687384214-19665-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786517687384214-19665-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:54:49.412631 19798 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 06:54:49.416251 19798 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 06:54:49.416605 19798 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:54:49.417475 19798 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 06:54:49.418746 19798 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 06:54:49.418823 19798 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 06:54:49.418891 19798 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 06:54:49.418921 19798 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 06:54:49.438832 19798 rpc_server.cc:307] RPC server started. Bound to: 127.19.52.65:41955
I20260812 06:54:49.439007 19941 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.52.65:41955 every 8 connection(s)
I20260812 06:54:49.439985 19798 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786517687384214-19665-0/minicluster-data/ts-0/data/info.pb
I20260812 06:54:49.442888 19665 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu as pid 19798
I20260812 06:54:49.442989 19665 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786517687384214-19665-0/minicluster-data/ts-0/wal/instance
I20260812 06:54:49.443310 19665 external_mini_cluster.cc:1428] Setting key a6e8037d69f0bd7fe1b64cfe769a2c32
Aug 12 06:54:49 dist-test-slave-65mx krb5kdc[19674](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517689, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.52.65@KRBTEST.COM for kudu/127.19.52.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.19.52.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.52.66@KRBTEST.COM" created.
I20260812 06:54:49.459232 19771 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:54:49.442633 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.52.65:52595 (local address 127.19.52.126:42493)
0812 06:54:49.442802 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0812 06:54:49.442808 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 06:54:49.443798 (+ 990us) server_negotiation.cc:407] Connection header received
0812 06:54:49.446570 (+ 2772us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:54:49.446574 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:54:49.446627 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:54:49.446714 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:54:49.448412 (+ 1698us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:54:49.449038 (+ 626us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:54:49.450383 (+ 1345us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:54:49.450640 (+ 257us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:54:49.450756 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:54:49.454489 (+ 3733us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:54:49.454497 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:54:49.454502 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:54:49.454541 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:54:49.456389 (+ 1848us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:54:49.457131 (+ 742us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:54:49.457134 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:54:49.457136 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:54:49.457233 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:54:49.457662 (+ 429us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:54:49.457673 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:54:49.457674 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:54:49.457901 (+ 227us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:54:49.457991 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0812 06:54:49.458751 (+ 760us) server_negotiation.cc:299] Negotiation successful
0812 06:54:49.458931 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260812 06:54:49.461678 19944 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:54:49.443039 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.126:42493 (local address 127.19.52.65:52595)
0812 06:54:49.443616 (+ 577us) negotiation.cc:107] Waiting for socket to connect
0812 06:54:49.443678 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0812 06:54:49.446289 (+ 2611us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:54:49.446944 (+ 655us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:54:49.446957 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:54:49.447604 (+ 647us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:54:49.448246 (+ 642us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:54:49.448261 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:54:49.449175 (+ 914us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:54:49.449178 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:54:49.450177 (+ 999us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:54:49.450187 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:54:49.450820 (+ 633us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:54:49.450902 (+ 82us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:54:49.451215 (+ 313us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:54:49.451980 (+ 765us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 06:54:49.451999 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:54:49.454219 (+ 2220us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 06:54:49.456622 (+ 2403us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:54:49.456670 (+ 48us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:54:49.456687 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:54:49.457007 (+ 320us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:54:49.457339 (+ 332us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:54:49.457341 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:54:49.457342 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:54:49.457500 (+ 158us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:54:49.458068 (+ 568us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 06:54:49.458075 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 06:54:49.458391 (+ 316us) client_negotiation.cc:769] Sending connection context
0812 06:54:49.458590 (+ 199us) client_negotiation.cc:240] Negotiation successful
0812 06:54:49.459000 (+ 410us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":324,"thread_start_us":132,"threads_started":1}
I20260812 06:54:49.463536 19942 heartbeater.cc:347] Connected to a master server at 127.19.52.126:42493
I20260812 06:54:49.463836 19942 heartbeater.cc:464] Registering TS with master...
I20260812 06:54:49.464402 19942 heartbeater.cc:511] Master 127.19.52.126:42493 requested a full tablet report, sending...
I20260812 06:54:49.466660 19727 ts_manager.cc:194] Registered new tserver with Master: 0e4f9bf6f286465aae2354003ada6727 (127.19.52.65:41955)
I20260812 06:54:49.468155 19727 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.52.65@KRBTEST.COM'} at 127.19.52.65:52595
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.52.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786517687384214-19665-0/krb5kdc/kudu_127.19.52.66.keytab.
Entry for principal kudu/127.19.52.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786517687384214-19665-0/krb5kdc/kudu_127.19.52.66.keytab.
I20260812 06:54:49.479478 19665 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu
/tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786517687384214-19665-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786517687384214-19665-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786517687384214-19665-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786517687384214-19665-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.52.66:0
--local_ip_for_outbound_sockets=127.19.52.66
--webserver_interface=127.19.52.66
--webserver_port=0
--tserver_master_addrs=127.19.52.126:42493
--keytab_file=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786517687384214-19665-0/krb5kdc/kudu_127.19.52.66.keytab
--principal=kudu/127.19.52.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.52.84:38137
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786517687384214-19665-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786517687384214-19665-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786517687384214-19665-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 06:54:49.627928 19947 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:54:49.628281 19947 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:54:49.628381 19947 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 06:54:49.632905 19947 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 06:54:49.633009 19947 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 06:54:49.633112 19947 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.52.66
I20260812 06:54:49.640281 19947 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.52.84:38137
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786517687384214-19665-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786517687384214-19665-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786517687384214-19665-0/krb5kdc/kudu_127.19.52.66.keytab
--principal=kudu/127.19.52.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.52.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786517687384214-19665-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.19.52.66
--webserver_port=0
--tserver_master_addrs=127.19.52.126:42493
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19947
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.52.66
--log_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786517687384214-19665-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 06:15:54 UTC on ccycloud.quasar-rnfzce.root.comops.site
build id 4581989
I20260812 06:54:49.641750 19947 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:54:49.642861 19947 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 06:54:49.654409 19956 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:54:49.654433 19953 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:54:49.654564 19952 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:54:49.654945 19947 server_base.cc:1034] running on GCE node
I20260812 06:54:49.655426 19947 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:54:49.656265 19947 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:54:49.657593 19947 hybrid_clock.cc:648] HybridClock initialized: now 1786517689657571 us; error 80 us; skew 500 ppm
Aug 12 06:54:49 dist-test-slave-65mx krb5kdc[19674](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517689, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.52.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 06:54:49.661278 19947 init.cc:377] Logged in from keytab as kudu/127.19.52.66@KRBTEST.COM (short username kudu)
I20260812 06:54:49.662236 19947 webserver.cc:460] Webserver started at http://127.19.52.66:42363/ using document root <none> and password file <none>
I20260812 06:54:49.662854 19947 fs_manager.cc:359] Metadata directory not provided
I20260812 06:54:49.662953 19947 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:54:49.663200 19947 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 06:54:49.665597 19947 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786517687384214-19665-0/minicluster-data/ts-1/data/instance:
uuid: "02fd50a110b84647bd5e3838500e8a2b"
format_stamp: "Formatted at 2026-08-12 06:54:49 on dist-test-slave-65mx"
server_key: "fa9275f7ed7e71a9990a9453432f2139"
server_key_iv: "3029bdbc8b69765fec97675e7cc266fc"
server_key_version: "encryptionkey@0"
I20260812 06:54:49.666239 19947 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786517687384214-19665-0/minicluster-data/ts-1/wal/instance:
uuid: "02fd50a110b84647bd5e3838500e8a2b"
format_stamp: "Formatted at 2026-08-12 06:54:49 on dist-test-slave-65mx"
server_key: "fa9275f7ed7e71a9990a9453432f2139"
server_key_iv: "3029bdbc8b69765fec97675e7cc266fc"
server_key_version: "encryptionkey@0"
I20260812 06:54:49.670972 19947 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260812 06:54:49.673803 19965 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 06:54:49.675210 19947 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 06:54:49.675383 19947 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786517687384214-19665-0/minicluster-data/ts-1/data,/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786517687384214-19665-0/minicluster-data/ts-1/wal
uuid: "02fd50a110b84647bd5e3838500e8a2b"
format_stamp: "Formatted at 2026-08-12 06:54:49 on dist-test-slave-65mx"
server_key: "fa9275f7ed7e71a9990a9453432f2139"
server_key_iv: "3029bdbc8b69765fec97675e7cc266fc"
server_key_version: "encryptionkey@0"
I20260812 06:54:49.675518 19947 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786517687384214-19665-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786517687384214-19665-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786517687384214-19665-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:54:49.692847 19947 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 06:54:49.696462 19947 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 06:54:49.696728 19947 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:54:49.697467 19947 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 06:54:49.698344 19947 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 06:54:49.698417 19947 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 06:54:49.698495 19947 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 06:54:49.698531 19947 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 06:54:49.714560 19947 rpc_server.cc:307] RPC server started. Bound to: 127.19.52.66:34329
I20260812 06:54:49.714613 20091 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.52.66:34329 every 8 connection(s)
I20260812 06:54:49.715638 19947 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786517687384214-19665-0/minicluster-data/ts-1/data/info.pb
I20260812 06:54:49.719455 19665 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu as pid 19947
I20260812 06:54:49.719610 19665 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786517687384214-19665-0/minicluster-data/ts-1/wal/instance
I20260812 06:54:49.719949 19665 external_mini_cluster.cc:1428] Setting key d0b85fddc7545b83b320be7969050b13
Aug 12 06:54:49 dist-test-slave-65mx krb5kdc[19674](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517689, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.52.66@KRBTEST.COM for kudu/127.19.52.126@KRBTEST.COM
I20260812 06:54:49.733436 19771 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:54:49.717859 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.52.66:44887 (local address 127.19.52.126:42493)
0812 06:54:49.718024 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0812 06:54:49.718029 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 06:54:49.719073 (+ 1044us) server_negotiation.cc:407] Connection header received
0812 06:54:49.720987 (+ 1914us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:54:49.720991 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:54:49.721041 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:54:49.721124 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:54:49.723094 (+ 1970us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:54:49.723634 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:54:49.725400 (+ 1766us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:54:49.725797 (+ 397us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:54:49.725931 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:54:49.729452 (+ 3521us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:54:49.729456 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:54:49.729458 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:54:49.729487 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:54:49.731089 (+ 1602us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:54:49.731714 (+ 625us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:54:49.731717 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:54:49.731718 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:54:49.731801 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:54:49.732207 (+ 406us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:54:49.732210 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:54:49.732211 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:54:49.732405 (+ 194us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:54:49.732518 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0812 06:54:49.733137 (+ 619us) server_negotiation.cc:299] Negotiation successful
0812 06:54:49.733266 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
WARNING: no policy specified for kudu/127.19.52.67@KRBTEST.COM; defaulting to no policy
I20260812 06:54:49.734380 20094 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:54:49.718334 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.126:42493 (local address 127.19.52.66:44887)
0812 06:54:49.718875 (+ 541us) negotiation.cc:107] Waiting for socket to connect
0812 06:54:49.718921 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0812 06:54:49.720694 (+ 1773us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:54:49.721336 (+ 642us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:54:49.721346 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:54:49.721962 (+ 616us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:54:49.722881 (+ 919us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:54:49.722905 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:54:49.723792 (+ 887us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:54:49.723799 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:54:49.725250 (+ 1451us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:54:49.725261 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:54:49.726045 (+ 784us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:54:49.726053 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:54:49.726315 (+ 262us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:54:49.727074 (+ 759us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 06:54:49.727101 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:54:49.729230 (+ 2129us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 06:54:49.731240 (+ 2010us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:54:49.731247 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:54:49.731267 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:54:49.731578 (+ 311us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:54:49.731903 (+ 325us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:54:49.731906 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:54:49.731908 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:54:49.732080 (+ 172us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:54:49.732571 (+ 491us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 06:54:49.732580 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 06:54:49.732873 (+ 293us) client_negotiation.cc:769] Sending connection context
0812 06:54:49.733074 (+ 201us) client_negotiation.cc:240] Negotiation successful
0812 06:54:49.733437 (+ 363us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":322,"thread_start_us":160,"threads_started":1}
I20260812 06:54:49.735798 20092 heartbeater.cc:347] Connected to a master server at 127.19.52.126:42493
I20260812 06:54:49.736093 20092 heartbeater.cc:464] Registering TS with master...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.52.67@KRBTEST.COM" created.
I20260812 06:54:49.736755 20092 heartbeater.cc:511] Master 127.19.52.126:42493 requested a full tablet report, sending...
I20260812 06:54:49.737920 19727 ts_manager.cc:194] Registered new tserver with Master: 02fd50a110b84647bd5e3838500e8a2b (127.19.52.66:34329)
I20260812 06:54:49.738545 19727 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.52.66@KRBTEST.COM'} at 127.19.52.66:44887
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.52.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786517687384214-19665-0/krb5kdc/kudu_127.19.52.67.keytab.
Entry for principal kudu/127.19.52.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786517687384214-19665-0/krb5kdc/kudu_127.19.52.67.keytab.
I20260812 06:54:49.756268 19665 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu
/tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786517687384214-19665-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786517687384214-19665-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786517687384214-19665-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786517687384214-19665-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.52.67:0
--local_ip_for_outbound_sockets=127.19.52.67
--webserver_interface=127.19.52.67
--webserver_port=0
--tserver_master_addrs=127.19.52.126:42493
--keytab_file=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786517687384214-19665-0/krb5kdc/kudu_127.19.52.67.keytab
--principal=kudu/127.19.52.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.52.84:38137
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786517687384214-19665-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786517687384214-19665-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786517687384214-19665-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 06:54:49.911692 20098 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:54:49.912061 20098 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:54:49.912149 20098 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 06:54:49.916594 20098 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 06:54:49.916713 20098 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 06:54:49.916813 20098 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.52.67
I20260812 06:54:49.922741 20098 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.52.84:38137
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786517687384214-19665-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786517687384214-19665-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786517687384214-19665-0/krb5kdc/kudu_127.19.52.67.keytab
--principal=kudu/127.19.52.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.52.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786517687384214-19665-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.19.52.67
--webserver_port=0
--tserver_master_addrs=127.19.52.126:42493
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20098
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.52.67
--log_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786517687384214-19665-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 06:15:54 UTC on ccycloud.quasar-rnfzce.root.comops.site
build id 4581989
I20260812 06:54:49.923926 20098 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:54:49.925071 20098 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 06:54:49.935927 20105 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:54:49.936087 20107 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:54:49.936204 20098 server_base.cc:1034] running on GCE node
W20260812 06:54:49.935927 20104 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:54:49.936895 20098 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:54:49.937675 20098 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:54:49.938887 20098 hybrid_clock.cc:648] HybridClock initialized: now 1786517689938850 us; error 47 us; skew 500 ppm
Aug 12 06:54:49 dist-test-slave-65mx krb5kdc[19674](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517689, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.52.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 06:54:49.942777 20098 init.cc:377] Logged in from keytab as kudu/127.19.52.67@KRBTEST.COM (short username kudu)
I20260812 06:54:49.943755 20098 webserver.cc:460] Webserver started at http://127.19.52.67:45897/ using document root <none> and password file <none>
I20260812 06:54:49.944341 20098 fs_manager.cc:359] Metadata directory not provided
I20260812 06:54:49.944408 20098 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:54:49.944679 20098 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 06:54:49.947202 20098 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786517687384214-19665-0/minicluster-data/ts-2/data/instance:
uuid: "55ae8528fb134f1dac8fe0e834837c18"
format_stamp: "Formatted at 2026-08-12 06:54:49 on dist-test-slave-65mx"
server_key: "adc482770bde7c83a1e46dcc3caf64e4"
server_key_iv: "e3105547fa8c085670aa72fbb44b4572"
server_key_version: "encryptionkey@0"
I20260812 06:54:49.947851 20098 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786517687384214-19665-0/minicluster-data/ts-2/wal/instance:
uuid: "55ae8528fb134f1dac8fe0e834837c18"
format_stamp: "Formatted at 2026-08-12 06:54:49 on dist-test-slave-65mx"
server_key: "adc482770bde7c83a1e46dcc3caf64e4"
server_key_iv: "e3105547fa8c085670aa72fbb44b4572"
server_key_version: "encryptionkey@0"
I20260812 06:54:49.952919 20098 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260812 06:54:49.956462 20114 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 06:54:49.958151 20098 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260812 06:54:49.958326 20098 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786517687384214-19665-0/minicluster-data/ts-2/data,/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786517687384214-19665-0/minicluster-data/ts-2/wal
uuid: "55ae8528fb134f1dac8fe0e834837c18"
format_stamp: "Formatted at 2026-08-12 06:54:49 on dist-test-slave-65mx"
server_key: "adc482770bde7c83a1e46dcc3caf64e4"
server_key_iv: "e3105547fa8c085670aa72fbb44b4572"
server_key_version: "encryptionkey@0"
I20260812 06:54:49.958479 20098 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786517687384214-19665-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786517687384214-19665-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786517687384214-19665-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:54:49.975744 20098 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 06:54:49.979537 20098 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 06:54:49.979821 20098 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:54:49.980748 20098 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 06:54:49.981825 20098 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 06:54:49.981881 20098 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 06:54:49.981926 20098 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 06:54:49.981943 20098 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 06:54:50.002164 20098 rpc_server.cc:307] RPC server started. Bound to: 127.19.52.67:41727
I20260812 06:54:50.002189 20238 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.52.67:41727 every 8 connection(s)
I20260812 06:54:50.003326 20098 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786517687384214-19665-0/minicluster-data/ts-2/data/info.pb
I20260812 06:54:50.006209 19665 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu as pid 20098
I20260812 06:54:50.006359 19665 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786517687384214-19665-0/minicluster-data/ts-2/wal/instance
I20260812 06:54:50.006676 19665 external_mini_cluster.cc:1428] Setting key 87eea85d21f456a98bce47e616854ece
Aug 12 06:54:50 dist-test-slave-65mx krb5kdc[19674](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517689, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.52.67@KRBTEST.COM for kudu/127.19.52.126@KRBTEST.COM
I20260812 06:54:50.019215 19771 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:54:50.005393 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.52.67:60685 (local address 127.19.52.126:42493)
0812 06:54:50.005569 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0812 06:54:50.005573 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 06:54:50.006769 (+ 1196us) server_negotiation.cc:407] Connection header received
0812 06:54:50.008596 (+ 1827us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:54:50.008599 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:54:50.008660 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:54:50.008772 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:54:50.010260 (+ 1488us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:54:50.010796 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:54:50.012113 (+ 1317us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:54:50.012335 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:54:50.012449 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:54:50.015403 (+ 2954us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:54:50.015407 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:54:50.015409 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:54:50.015442 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:54:50.017041 (+ 1599us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:54:50.017624 (+ 583us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:54:50.017627 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:54:50.017629 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:54:50.017709 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:54:50.018078 (+ 369us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:54:50.018081 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:54:50.018083 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:54:50.018280 (+ 197us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:54:50.018391 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0812 06:54:50.018914 (+ 523us) server_negotiation.cc:299] Negotiation successful
0812 06:54:50.019046 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260812 06:54:50.020155 20242 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:54:50.005816 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.126:42493 (local address 127.19.52.67:60685)
0812 06:54:50.006606 (+ 790us) negotiation.cc:107] Waiting for socket to connect
0812 06:54:50.006655 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0812 06:54:50.008339 (+ 1684us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:54:50.008959 (+ 620us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:54:50.008966 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:54:50.009492 (+ 526us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:54:50.010108 (+ 616us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:54:50.010123 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:54:50.010928 (+ 805us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:54:50.010936 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:54:50.011972 (+ 1036us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:54:50.011982 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:54:50.012493 (+ 511us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:54:50.012497 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:54:50.012743 (+ 246us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:54:50.013476 (+ 733us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 06:54:50.013493 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:54:50.015215 (+ 1722us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 06:54:50.017180 (+ 1965us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:54:50.017186 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:54:50.017206 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:54:50.017484 (+ 278us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:54:50.017809 (+ 325us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:54:50.017812 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:54:50.017814 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:54:50.017959 (+ 145us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:54:50.018392 (+ 433us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 06:54:50.018400 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 06:54:50.018665 (+ 265us) client_negotiation.cc:769] Sending connection context
0812 06:54:50.018870 (+ 205us) client_negotiation.cc:240] Negotiation successful
0812 06:54:50.019230 (+ 360us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":544,"thread_start_us":294,"threads_started":1}
I20260812 06:54:50.021363 20239 heartbeater.cc:347] Connected to a master server at 127.19.52.126:42493
I20260812 06:54:50.021642 20239 heartbeater.cc:464] Registering TS with master...
I20260812 06:54:50.022253 20239 heartbeater.cc:511] Master 127.19.52.126:42493 requested a full tablet report, sending...
I20260812 06:54:50.023376 19726 ts_manager.cc:194] Registered new tserver with Master: 55ae8528fb134f1dac8fe0e834837c18 (127.19.52.67:41727)
I20260812 06:54:50.024026 19726 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.52.67@KRBTEST.COM'} at 127.19.52.67:60685
I20260812 06:54:50.034448 19665 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 12 06:54:50 dist-test-slave-65mx krb5kdc[19674](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517690, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 12 06:54:50 dist-test-slave-65mx krb5kdc[19674](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517690, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.19.52.65@KRBTEST.COM
I20260812 06:54:50.067485 20247 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:54:50.055702 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34436 (local address 127.19.52.65:41955)
0812 06:54:50.056056 (+ 354us) server_negotiation.cc:206] Beginning negotiation
0812 06:54:50.056067 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0812 06:54:50.056128 (+ 61us) server_negotiation.cc:407] Connection header received
0812 06:54:50.056315 (+ 187us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:54:50.056320 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:54:50.056508 (+ 188us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:54:50.056712 (+ 204us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:54:50.057562 (+ 850us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:54:50.058387 (+ 825us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:54:50.059347 (+ 960us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:54:50.059772 (+ 425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:54:50.059902 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:54:50.063235 (+ 3333us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:54:50.063248 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:54:50.063262 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:54:50.063292 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:54:50.065300 (+ 2008us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:54:50.065847 (+ 547us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:54:50.065853 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:54:50.065855 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:54:50.065993 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:54:50.066310 (+ 317us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:54:50.066314 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:54:50.066316 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:54:50.066648 (+ 332us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:54:50.066844 (+ 196us) server_negotiation.cc:1035] Waiting for connection context
0812 06:54:50.067045 (+ 201us) server_negotiation.cc:299] Negotiation successful
0812 06:54:50.067194 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":245,"thread_start_us":110,"threads_started":1}
W20260812 06:54:50.068908 19847 server_base.cc:1116] Unauthorized access attempt to method kudu.tserver.TabletServerService.ListTablets from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:34436
Aug 12 06:54:50 dist-test-slave-65mx krb5kdc[19674](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517690, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 12 06:54:50 dist-test-slave-65mx krb5kdc[19674](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517690, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.19.52.65@KRBTEST.COM
I20260812 06:54:50.099718 20247 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:54:50.090890 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34448 (local address 127.19.52.65:41955)
0812 06:54:50.091036 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0812 06:54:50.091056 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0812 06:54:50.091305 (+ 249us) server_negotiation.cc:407] Connection header received
0812 06:54:50.091491 (+ 186us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:54:50.091494 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:54:50.091540 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:54:50.091627 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:54:50.092372 (+ 745us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:54:50.092958 (+ 586us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:54:50.093810 (+ 852us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:54:50.094037 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:54:50.094158 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:54:50.096605 (+ 2447us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:54:50.096613 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:54:50.096616 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:54:50.096646 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:54:50.098032 (+ 1386us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:54:50.098470 (+ 438us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:54:50.098474 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:54:50.098476 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:54:50.098561 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:54:50.098846 (+ 285us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:54:50.098850 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:54:50.098852 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:54:50.099031 (+ 179us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:54:50.099185 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0812 06:54:50.099419 (+ 234us) server_negotiation.cc:299] Negotiation successful
0812 06:54:50.099547 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260812 06:54:50.101080 19665 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu with pid 19798
I20260812 06:54:50.111563 19665 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu with pid 19947
I20260812 06:54:50.120354 19665 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu with pid 20098
I20260812 06:54:50.128965 19665 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu with pid 19688
2026-08-12T06:54:50Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnListTablets (2709 ms)
[ RUN ] SecurityITest.TestAuthorizationOnChecksum
Loading random data
Initializing database '/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786517687384214-19665-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 06:54:50 dist-test-slave-65mx krb5kdc[20255](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 06:54:50 dist-test-slave-65mx krb5kdc[20255](info): set up 2 sockets
Aug 12 06:54:50 dist-test-slave-65mx krb5kdc[20255](info): commencing operation
krb5kdc: starting...
W20260812 06:54:51.266927 19665 mini_kdc.cc:121] Time spent starting KDC: real 1.111s user 0.006s sys 0.001s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786517687384214-19665-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786517687384214-19665-0/krb5kdc/test-user.keytab.
Aug 12 06:54:51 dist-test-slave-65mx krb5kdc[20255](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517691, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T06:54:51Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T06:54:51Z Disabled control of system clock
WARNING: no policy specified for kudu/127.19.52.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.52.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.52.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786517687384214-19665-0/krb5kdc/kudu_127.19.52.126.keytab.
Entry for principal kudu/127.19.52.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786517687384214-19665-0/krb5kdc/kudu_127.19.52.126.keytab.
I20260812 06:54:51.407781 19665 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu
/tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786517687384214-19665-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786517687384214-19665-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786517687384214-19665-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786517687384214-19665-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.52.126:33709
--webserver_interface=127.19.52.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786517687384214-19665-0/krb5kdc/kudu_127.19.52.126.keytab
--principal=kudu/127.19.52.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.52.84:44333
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.19.52.126:33709
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786517687384214-19665-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786517687384214-19665-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786517687384214-19665-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 06:54:51.533103 20269 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:54:51.533396 20269 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:54:51.533439 20269 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 06:54:51.537583 20269 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 06:54:51.537657 20269 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 06:54:51.537678 20269 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 06:54:51.537696 20269 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 06:54:51.537712 20269 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 06:54:51.543443 20269 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.52.84:44333
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786517687384214-19665-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786517687384214-19665-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.19.52.126:33709
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786517687384214-19665-0/krb5kdc/kudu_127.19.52.126.keytab
--principal=kudu/127.19.52.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.52.126:33709
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786517687384214-19665-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.19.52.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20269
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786517687384214-19665-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 06:15:54 UTC on ccycloud.quasar-rnfzce.root.comops.site
build id 4581989
I20260812 06:54:51.544626 20269 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:54:51.545672 20269 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 06:54:51.554486 20276 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:54:51.554518 20269 server_base.cc:1034] running on GCE node
W20260812 06:54:51.554462 20278 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:54:51.554466 20275 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:54:51.555291 20269 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:54:51.556511 20269 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:54:51.557773 20269 hybrid_clock.cc:648] HybridClock initialized: now 1786517691557755 us; error 28 us; skew 500 ppm
Aug 12 06:54:51 dist-test-slave-65mx krb5kdc[20255](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517691, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.52.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 06:54:51.561209 20269 init.cc:377] Logged in from keytab as kudu/127.19.52.126@KRBTEST.COM (short username kudu)
I20260812 06:54:51.562170 20269 webserver.cc:460] Webserver started at http://127.19.52.126:37861/ using document root <none> and password file <none>
I20260812 06:54:51.562752 20269 fs_manager.cc:359] Metadata directory not provided
I20260812 06:54:51.562816 20269 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:54:51.563071 20269 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 06:54:51.565300 20269 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786517687384214-19665-0/minicluster-data/master-0/data/instance:
uuid: "3e7c07792d294e319fb7dcf6c1959ede"
format_stamp: "Formatted at 2026-08-12 06:54:51 on dist-test-slave-65mx"
server_key: "ec22670d98c4ae8c581f42ec495c390a"
server_key_iv: "0ef00cedad5ae6688a7a0e984f5766e2"
server_key_version: "encryptionkey@0"
I20260812 06:54:51.565889 20269 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786517687384214-19665-0/minicluster-data/master-0/wal/instance:
uuid: "3e7c07792d294e319fb7dcf6c1959ede"
format_stamp: "Formatted at 2026-08-12 06:54:51 on dist-test-slave-65mx"
server_key: "ec22670d98c4ae8c581f42ec495c390a"
server_key_iv: "0ef00cedad5ae6688a7a0e984f5766e2"
server_key_version: "encryptionkey@0"
I20260812 06:54:51.569808 20269 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260812 06:54:51.572379 20286 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 06:54:51.573871 20269 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 06:54:51.574059 20269 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786517687384214-19665-0/minicluster-data/master-0/data,/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786517687384214-19665-0/minicluster-data/master-0/wal
uuid: "3e7c07792d294e319fb7dcf6c1959ede"
format_stamp: "Formatted at 2026-08-12 06:54:51 on dist-test-slave-65mx"
server_key: "ec22670d98c4ae8c581f42ec495c390a"
server_key_iv: "0ef00cedad5ae6688a7a0e984f5766e2"
server_key_version: "encryptionkey@0"
I20260812 06:54:51.574187 20269 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786517687384214-19665-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786517687384214-19665-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786517687384214-19665-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:54:51.608395 20269 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 06:54:51.611982 20269 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 06:54:51.612164 20269 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:54:51.620188 20269 rpc_server.cc:307] RPC server started. Bound to: 127.19.52.126:33709
I20260812 06:54:51.620213 20349 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.52.126:33709 every 8 connection(s)
I20260812 06:54:51.621273 20269 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786517687384214-19665-0/minicluster-data/master-0/data/info.pb
I20260812 06:54:51.624491 20352 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 06:54:51.627638 19665 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu as pid 20269
I20260812 06:54:51.627789 19665 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786517687384214-19665-0/minicluster-data/master-0/wal/instance
I20260812 06:54:51.628093 19665 external_mini_cluster.cc:1428] Setting key c6084d27b2ee84a6723568c663761320
I20260812 06:54:51.631031 20352 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3e7c07792d294e319fb7dcf6c1959ede: Bootstrap starting.
I20260812 06:54:51.633754 20352 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3e7c07792d294e319fb7dcf6c1959ede: Neither blocks nor log segments found. Creating new log.
I20260812 06:54:51.634680 20352 log.cc:824] T 00000000000000000000000000000000 P 3e7c07792d294e319fb7dcf6c1959ede: Log is configured to *not* fsync() on all Append() calls
I20260812 06:54:51.637329 20352 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3e7c07792d294e319fb7dcf6c1959ede: No bootstrap required, opened a new log
Aug 12 06:54:51 dist-test-slave-65mx krb5kdc[20255](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517691, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.19.52.126@KRBTEST.COM
I20260812 06:54:51.641700 20352 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3e7c07792d294e319fb7dcf6c1959ede [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3e7c07792d294e319fb7dcf6c1959ede" member_type: VOTER last_known_addr { host: "127.19.52.126" port: 33709 } }
I20260812 06:54:51.641968 20352 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3e7c07792d294e319fb7dcf6c1959ede [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 06:54:51.642004 20352 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3e7c07792d294e319fb7dcf6c1959ede [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3e7c07792d294e319fb7dcf6c1959ede, State: Initialized, Role: FOLLOWER
I20260812 06:54:51.642486 20352 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3e7c07792d294e319fb7dcf6c1959ede [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3e7c07792d294e319fb7dcf6c1959ede" member_type: VOTER last_known_addr { host: "127.19.52.126" port: 33709 } }
I20260812 06:54:51.642627 20352 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3e7c07792d294e319fb7dcf6c1959ede [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 06:54:51.642715 20352 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3e7c07792d294e319fb7dcf6c1959ede [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 06:54:51.642822 20352 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3e7c07792d294e319fb7dcf6c1959ede [term 0 FOLLOWER]: Advancing to term 1
I20260812 06:54:51.644095 20352 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3e7c07792d294e319fb7dcf6c1959ede [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3e7c07792d294e319fb7dcf6c1959ede" member_type: VOTER last_known_addr { host: "127.19.52.126" port: 33709 } }
I20260812 06:54:51.644459 20352 leader_election.cc:302] T 00000000000000000000000000000000 P 3e7c07792d294e319fb7dcf6c1959ede [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3e7c07792d294e319fb7dcf6c1959ede; no voters:
I20260812 06:54:51.644906 20352 leader_election.cc:288] T 00000000000000000000000000000000 P 3e7c07792d294e319fb7dcf6c1959ede [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 06:54:51.645074 20363 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3e7c07792d294e319fb7dcf6c1959ede [term 1 FOLLOWER]: Leader election won for term 1
I20260812 06:54:51.645409 20363 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3e7c07792d294e319fb7dcf6c1959ede [term 1 LEADER]: Becoming Leader. State: Replica: 3e7c07792d294e319fb7dcf6c1959ede, State: Running, Role: LEADER
I20260812 06:54:51.645925 20363 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3e7c07792d294e319fb7dcf6c1959ede [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3e7c07792d294e319fb7dcf6c1959ede" member_type: VOTER last_known_addr { host: "127.19.52.126" port: 33709 } }
I20260812 06:54:51.646289 20352 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3e7c07792d294e319fb7dcf6c1959ede [sys.catalog]: configured and running, proceeding with master startup.
I20260812 06:54:51.646723 20361 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:54:51.629923 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40684 (local address 127.19.52.126:33709)
0812 06:54:51.630477 (+ 554us) server_negotiation.cc:206] Beginning negotiation
0812 06:54:51.630500 (+ 23us) server_negotiation.cc:399] Waiting for connection header
0812 06:54:51.630538 (+ 38us) server_negotiation.cc:407] Connection header received
0812 06:54:51.631229 (+ 691us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:54:51.631255 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:54:51.631617 (+ 362us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:54:51.631905 (+ 288us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:54:51.632986 (+ 1081us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:54:51.634230 (+ 1244us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:54:51.635390 (+ 1160us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:54:51.636013 (+ 623us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:54:51.636179 (+ 166us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:54:51.639338 (+ 3159us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:54:51.639343 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:54:51.639359 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:54:51.639393 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:54:51.642777 (+ 3384us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:54:51.643403 (+ 626us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:54:51.643409 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:54:51.643414 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:54:51.643585 (+ 171us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:54:51.643939 (+ 354us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:54:51.643942 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:54:51.643944 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:54:51.644477 (+ 533us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:54:51.644768 (+ 291us) server_negotiation.cc:1035] Waiting for connection context
0812 06:54:51.645191 (+ 423us) server_negotiation.cc:299] Negotiation successful
0812 06:54:51.645700 (+ 509us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":383,"thread_start_us":171,"threads_started":1}
I20260812 06:54:51.647275 20363 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3e7c07792d294e319fb7dcf6c1959ede [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3e7c07792d294e319fb7dcf6c1959ede. Latest consensus state: current_term: 1 leader_uuid: "3e7c07792d294e319fb7dcf6c1959ede" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3e7c07792d294e319fb7dcf6c1959ede" member_type: VOTER last_known_addr { host: "127.19.52.126" port: 33709 } } }
I20260812 06:54:51.647469 20363 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3e7c07792d294e319fb7dcf6c1959ede [sys.catalog]: This master's current role is: LEADER
I20260812 06:54:51.647748 20364 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3e7c07792d294e319fb7dcf6c1959ede [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3e7c07792d294e319fb7dcf6c1959ede" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3e7c07792d294e319fb7dcf6c1959ede" member_type: VOTER last_known_addr { host: "127.19.52.126" port: 33709 } } }
I20260812 06:54:51.647913 20364 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3e7c07792d294e319fb7dcf6c1959ede [sys.catalog]: This master's current role is: LEADER
I20260812 06:54:51.647910 20372 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 06:54:51.651996 20372 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 06:54:51.657809 20372 catalog_manager.cc:1347] Generated new cluster ID: 2a19507617324c08a16b7f499abe3f0c
I20260812 06:54:51.657915 20372 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 06:54:51.678020 20372 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 06:54:51.679388 20372 catalog_manager.cc:1504] Loading token signing keys...
I20260812 06:54:51.687762 20372 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3e7c07792d294e319fb7dcf6c1959ede: Generated new TSK 0
I20260812 06:54:51.688710 20372 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.19.52.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.52.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.52.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786517687384214-19665-0/krb5kdc/kudu_127.19.52.65.keytab.
Entry for principal kudu/127.19.52.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786517687384214-19665-0/krb5kdc/kudu_127.19.52.65.keytab.
I20260812 06:54:51.728757 19665 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu
/tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786517687384214-19665-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786517687384214-19665-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786517687384214-19665-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786517687384214-19665-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.52.65:0
--local_ip_for_outbound_sockets=127.19.52.65
--webserver_interface=127.19.52.65
--webserver_port=0
--tserver_master_addrs=127.19.52.126:33709
--keytab_file=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786517687384214-19665-0/krb5kdc/kudu_127.19.52.65.keytab
--principal=kudu/127.19.52.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.52.84:44333
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786517687384214-19665-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786517687384214-19665-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786517687384214-19665-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 06:54:51.851864 20385 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:54:51.852173 20385 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:54:51.852223 20385 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 06:54:51.856756 20385 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 06:54:51.856837 20385 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 06:54:51.856917 20385 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.52.65
I20260812 06:54:51.862761 20385 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.52.84:44333
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786517687384214-19665-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786517687384214-19665-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786517687384214-19665-0/krb5kdc/kudu_127.19.52.65.keytab
--principal=kudu/127.19.52.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.52.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786517687384214-19665-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.19.52.65
--webserver_port=0
--tserver_master_addrs=127.19.52.126:33709
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20385
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.52.65
--log_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786517687384214-19665-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 06:15:54 UTC on ccycloud.quasar-rnfzce.root.comops.site
build id 4581989
I20260812 06:54:51.864078 20385 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:54:51.865228 20385 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 06:54:51.875905 20394 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:54:51.875859 20392 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:54:51.875873 20391 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:54:51.876114 20385 server_base.cc:1034] running on GCE node
I20260812 06:54:51.876770 20385 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:54:51.877584 20385 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:54:51.878815 20385 hybrid_clock.cc:648] HybridClock initialized: now 1786517691878767 us; error 60 us; skew 500 ppm
Aug 12 06:54:51 dist-test-slave-65mx krb5kdc[20255](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517691, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.52.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 06:54:51.882027 20385 init.cc:377] Logged in from keytab as kudu/127.19.52.65@KRBTEST.COM (short username kudu)
I20260812 06:54:51.883047 20385 webserver.cc:460] Webserver started at http://127.19.52.65:41671/ using document root <none> and password file <none>
I20260812 06:54:51.883688 20385 fs_manager.cc:359] Metadata directory not provided
I20260812 06:54:51.883795 20385 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:54:51.884039 20385 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 06:54:51.886282 20385 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786517687384214-19665-0/minicluster-data/ts-0/data/instance:
uuid: "ef158262d7574ab998d2ead0bf1f300d"
format_stamp: "Formatted at 2026-08-12 06:54:51 on dist-test-slave-65mx"
server_key: "71a3aa5cf783db1defab10d008d48f6e"
server_key_iv: "84e4ce2e57da0b9c121a08b52a121b3e"
server_key_version: "encryptionkey@0"
I20260812 06:54:51.886883 20385 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786517687384214-19665-0/minicluster-data/ts-0/wal/instance:
uuid: "ef158262d7574ab998d2ead0bf1f300d"
format_stamp: "Formatted at 2026-08-12 06:54:51 on dist-test-slave-65mx"
server_key: "71a3aa5cf783db1defab10d008d48f6e"
server_key_iv: "84e4ce2e57da0b9c121a08b52a121b3e"
server_key_version: "encryptionkey@0"
I20260812 06:54:51.891196 20385 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260812 06:54:51.894119 20401 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 06:54:51.895579 20385 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260812 06:54:51.895749 20385 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786517687384214-19665-0/minicluster-data/ts-0/data,/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786517687384214-19665-0/minicluster-data/ts-0/wal
uuid: "ef158262d7574ab998d2ead0bf1f300d"
format_stamp: "Formatted at 2026-08-12 06:54:51 on dist-test-slave-65mx"
server_key: "71a3aa5cf783db1defab10d008d48f6e"
server_key_iv: "84e4ce2e57da0b9c121a08b52a121b3e"
server_key_version: "encryptionkey@0"
I20260812 06:54:51.895891 20385 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786517687384214-19665-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786517687384214-19665-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786517687384214-19665-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:54:51.910566 20385 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 06:54:51.914072 20385 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 06:54:51.914300 20385 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:54:51.914937 20385 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 06:54:51.915822 20385 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 06:54:51.915895 20385 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 06:54:51.915969 20385 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 06:54:51.916023 20385 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 06:54:51.926386 20385 rpc_server.cc:307] RPC server started. Bound to: 127.19.52.65:37203
I20260812 06:54:51.926410 20528 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.52.65:37203 every 8 connection(s)
I20260812 06:54:51.927356 20385 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786517687384214-19665-0/minicluster-data/ts-0/data/info.pb
I20260812 06:54:51.937558 19665 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu as pid 20385
I20260812 06:54:51.937709 19665 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786517687384214-19665-0/minicluster-data/ts-0/wal/instance
Aug 12 06:54:51 dist-test-slave-65mx krb5kdc[20255](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517691, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.52.65@KRBTEST.COM for kudu/127.19.52.126@KRBTEST.COM
I20260812 06:54:51.938054 19665 external_mini_cluster.cc:1428] Setting key 5b898076dda9f137c5813afa22fea544
I20260812 06:54:51.943580 20361 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:54:51.929492 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.52.65:44561 (local address 127.19.52.126:33709)
0812 06:54:51.929670 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0812 06:54:51.929676 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 06:54:51.930763 (+ 1087us) server_negotiation.cc:407] Connection header received
0812 06:54:51.932165 (+ 1402us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:54:51.932168 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:54:51.932218 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:54:51.932301 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:54:51.933617 (+ 1316us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:54:51.934174 (+ 557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:54:51.935368 (+ 1194us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:54:51.935640 (+ 272us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:54:51.935730 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:54:51.938694 (+ 2964us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:54:51.938698 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:54:51.938700 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:54:51.938731 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:54:51.940619 (+ 1888us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:54:51.941483 (+ 864us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:54:51.941486 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:54:51.941489 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:54:51.941589 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:54:51.942091 (+ 502us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:54:51.942094 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:54:51.942095 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:54:51.942417 (+ 322us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:54:51.942535 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0812 06:54:51.943182 (+ 647us) server_negotiation.cc:299] Negotiation successful
0812 06:54:51.943360 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260812 06:54:51.944823 20531 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:54:51.930033 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.126:33709 (local address 127.19.52.65:44561)
0812 06:54:51.930584 (+ 551us) negotiation.cc:107] Waiting for socket to connect
0812 06:54:51.930633 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0812 06:54:51.931903 (+ 1270us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:54:51.932488 (+ 585us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:54:51.932495 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:54:51.932899 (+ 404us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:54:51.933433 (+ 534us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:54:51.933447 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:54:51.934320 (+ 873us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:54:51.934323 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:54:51.935226 (+ 903us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:54:51.935237 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:54:51.935800 (+ 563us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:54:51.935802 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:54:51.935995 (+ 193us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:54:51.936702 (+ 707us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 06:54:51.936719 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:54:51.938408 (+ 1689us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 06:54:51.940776 (+ 2368us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:54:51.940787 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:54:51.940812 (+ 25us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:54:51.941340 (+ 528us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:54:51.941699 (+ 359us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:54:51.941706 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:54:51.941711 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:54:51.941962 (+ 251us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:54:51.942560 (+ 598us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 06:54:51.942566 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 06:54:51.942845 (+ 279us) client_negotiation.cc:769] Sending connection context
0812 06:54:51.943093 (+ 248us) client_negotiation.cc:240] Negotiation successful
0812 06:54:51.943396 (+ 303us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":343,"thread_start_us":148,"threads_started":1}
I20260812 06:54:51.946583 20529 heartbeater.cc:347] Connected to a master server at 127.19.52.126:33709
I20260812 06:54:51.946940 20529 heartbeater.cc:464] Registering TS with master...
I20260812 06:54:51.947607 20529 heartbeater.cc:511] Master 127.19.52.126:33709 requested a full tablet report, sending...
I20260812 06:54:51.949362 20307 ts_manager.cc:194] Registered new tserver with Master: ef158262d7574ab998d2ead0bf1f300d (127.19.52.65:37203)
WARNING: no policy specified for kudu/127.19.52.66@KRBTEST.COM; defaulting to no policy
I20260812 06:54:51.950635 20307 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.52.65@KRBTEST.COM'} at 127.19.52.65:44561
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.52.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.52.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786517687384214-19665-0/krb5kdc/kudu_127.19.52.66.keytab.
Entry for principal kudu/127.19.52.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786517687384214-19665-0/krb5kdc/kudu_127.19.52.66.keytab.
I20260812 06:54:51.972440 19665 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu
/tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786517687384214-19665-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786517687384214-19665-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786517687384214-19665-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786517687384214-19665-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.52.66:0
--local_ip_for_outbound_sockets=127.19.52.66
--webserver_interface=127.19.52.66
--webserver_port=0
--tserver_master_addrs=127.19.52.126:33709
--keytab_file=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786517687384214-19665-0/krb5kdc/kudu_127.19.52.66.keytab
--principal=kudu/127.19.52.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.52.84:44333
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786517687384214-19665-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786517687384214-19665-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786517687384214-19665-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 06:54:52.093225 20536 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:54:52.093542 20536 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:54:52.093590 20536 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 06:54:52.097828 20536 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 06:54:52.097909 20536 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 06:54:52.097976 20536 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.52.66
I20260812 06:54:52.103840 20536 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.52.84:44333
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786517687384214-19665-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786517687384214-19665-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786517687384214-19665-0/krb5kdc/kudu_127.19.52.66.keytab
--principal=kudu/127.19.52.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.52.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786517687384214-19665-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.19.52.66
--webserver_port=0
--tserver_master_addrs=127.19.52.126:33709
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20536
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.52.66
--log_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786517687384214-19665-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 06:15:54 UTC on ccycloud.quasar-rnfzce.root.comops.site
build id 4581989
I20260812 06:54:52.105031 20536 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:54:52.106063 20536 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 06:54:52.116465 20542 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:54:52.116423 20541 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:54:52.116417 20545 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:54:52.116852 20536 server_base.cc:1034] running on GCE node
I20260812 06:54:52.117332 20536 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:54:52.118026 20536 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:54:52.119217 20536 hybrid_clock.cc:648] HybridClock initialized: now 1786517692119181 us; error 51 us; skew 500 ppm
Aug 12 06:54:52 dist-test-slave-65mx krb5kdc[20255](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517692, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.52.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 06:54:52.122328 20536 init.cc:377] Logged in from keytab as kudu/127.19.52.66@KRBTEST.COM (short username kudu)
I20260812 06:54:52.123365 20536 webserver.cc:460] Webserver started at http://127.19.52.66:34613/ using document root <none> and password file <none>
I20260812 06:54:52.123975 20536 fs_manager.cc:359] Metadata directory not provided
I20260812 06:54:52.124075 20536 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:54:52.124331 20536 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 06:54:52.126598 20536 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786517687384214-19665-0/minicluster-data/ts-1/data/instance:
uuid: "ff835b4f96b849cf934c9ea972c47293"
format_stamp: "Formatted at 2026-08-12 06:54:52 on dist-test-slave-65mx"
server_key: "de50594c6f90e04690ad4183437ac74d"
server_key_iv: "8d54e792f6008d425a519cac87f85706"
server_key_version: "encryptionkey@0"
I20260812 06:54:52.127235 20536 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786517687384214-19665-0/minicluster-data/ts-1/wal/instance:
uuid: "ff835b4f96b849cf934c9ea972c47293"
format_stamp: "Formatted at 2026-08-12 06:54:52 on dist-test-slave-65mx"
server_key: "de50594c6f90e04690ad4183437ac74d"
server_key_iv: "8d54e792f6008d425a519cac87f85706"
server_key_version: "encryptionkey@0"
I20260812 06:54:52.131533 20536 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.003s
I20260812 06:54:52.135547 20555 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 06:54:52.137887 20536 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.000s
I20260812 06:54:52.138077 20536 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786517687384214-19665-0/minicluster-data/ts-1/data,/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786517687384214-19665-0/minicluster-data/ts-1/wal
uuid: "ff835b4f96b849cf934c9ea972c47293"
format_stamp: "Formatted at 2026-08-12 06:54:52 on dist-test-slave-65mx"
server_key: "de50594c6f90e04690ad4183437ac74d"
server_key_iv: "8d54e792f6008d425a519cac87f85706"
server_key_version: "encryptionkey@0"
I20260812 06:54:52.138263 20536 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786517687384214-19665-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786517687384214-19665-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786517687384214-19665-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:54:52.178741 20536 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 06:54:52.182488 20536 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 06:54:52.182742 20536 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:54:52.183475 20536 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 06:54:52.184415 20536 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 06:54:52.184469 20536 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 06:54:52.184512 20536 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 06:54:52.184607 20536 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 06:54:52.195050 20536 rpc_server.cc:307] RPC server started. Bound to: 127.19.52.66:37985
I20260812 06:54:52.195093 20688 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.52.66:37985 every 8 connection(s)
I20260812 06:54:52.196045 20536 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786517687384214-19665-0/minicluster-data/ts-1/data/info.pb
I20260812 06:54:52.201748 19665 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu as pid 20536
I20260812 06:54:52.201877 19665 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786517687384214-19665-0/minicluster-data/ts-1/wal/instance
I20260812 06:54:52.202169 19665 external_mini_cluster.cc:1428] Setting key f47a736645baca6cba876ba96950ed67
Aug 12 06:54:52 dist-test-slave-65mx krb5kdc[20255](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517692, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.52.66@KRBTEST.COM for kudu/127.19.52.126@KRBTEST.COM
I20260812 06:54:52.213994 20361 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:54:52.198263 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.52.66:56633 (local address 127.19.52.126:33709)
0812 06:54:52.198602 (+ 339us) server_negotiation.cc:206] Beginning negotiation
0812 06:54:52.198609 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 06:54:52.199370 (+ 761us) server_negotiation.cc:407] Connection header received
0812 06:54:52.201462 (+ 2092us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:54:52.201465 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:54:52.201519 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:54:52.201587 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:54:52.203237 (+ 1650us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:54:52.203908 (+ 671us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:54:52.205759 (+ 1851us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:54:52.205976 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:54:52.206136 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:54:52.209251 (+ 3115us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:54:52.209254 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:54:52.209257 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:54:52.209285 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:54:52.211095 (+ 1810us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:54:52.211958 (+ 863us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:54:52.211961 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:54:52.211962 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:54:52.212055 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:54:52.212630 (+ 575us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:54:52.212633 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:54:52.212635 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:54:52.212843 (+ 208us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:54:52.212927 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0812 06:54:52.213729 (+ 802us) server_negotiation.cc:299] Negotiation successful
0812 06:54:52.213857 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":35,"server-negotiator.queue_time_us":71}
WARNING: no policy specified for kudu/127.19.52.67@KRBTEST.COM; defaulting to no policy
I20260812 06:54:52.215092 20692 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:54:52.198640 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.126:33709 (local address 127.19.52.66:56633)
0812 06:54:52.199193 (+ 553us) negotiation.cc:107] Waiting for socket to connect
0812 06:54:52.199243 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0812 06:54:52.201181 (+ 1938us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:54:52.201847 (+ 666us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:54:52.201858 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:54:52.202404 (+ 546us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:54:52.203074 (+ 670us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:54:52.203094 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:54:52.204051 (+ 957us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:54:52.204055 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:54:52.205612 (+ 1557us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:54:52.205628 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:54:52.206186 (+ 558us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:54:52.206191 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:54:52.206377 (+ 186us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:54:52.207096 (+ 719us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 06:54:52.207114 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:54:52.209005 (+ 1891us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 06:54:52.211267 (+ 2262us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:54:52.211279 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:54:52.211304 (+ 25us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:54:52.211782 (+ 478us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:54:52.212163 (+ 381us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:54:52.212168 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:54:52.212170 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:54:52.212429 (+ 259us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:54:52.212968 (+ 539us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 06:54:52.212979 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 06:54:52.213393 (+ 414us) client_negotiation.cc:769] Sending connection context
0812 06:54:52.213655 (+ 262us) client_negotiation.cc:240] Negotiation successful
0812 06:54:52.214074 (+ 419us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":317,"thread_start_us":139,"threads_started":1}
I20260812 06:54:52.216610 20690 heartbeater.cc:347] Connected to a master server at 127.19.52.126:33709
I20260812 06:54:52.216883 20690 heartbeater.cc:464] Registering TS with master...
I20260812 06:54:52.217537 20690 heartbeater.cc:511] Master 127.19.52.126:33709 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.52.67@KRBTEST.COM" created.
I20260812 06:54:52.218854 20307 ts_manager.cc:194] Registered new tserver with Master: ff835b4f96b849cf934c9ea972c47293 (127.19.52.66:37985)
I20260812 06:54:52.219844 20307 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.52.66@KRBTEST.COM'} at 127.19.52.66:56633
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.52.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786517687384214-19665-0/krb5kdc/kudu_127.19.52.67.keytab.
Entry for principal kudu/127.19.52.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786517687384214-19665-0/krb5kdc/kudu_127.19.52.67.keytab.
I20260812 06:54:52.237563 19665 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu
/tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786517687384214-19665-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786517687384214-19665-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786517687384214-19665-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786517687384214-19665-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.52.67:0
--local_ip_for_outbound_sockets=127.19.52.67
--webserver_interface=127.19.52.67
--webserver_port=0
--tserver_master_addrs=127.19.52.126:33709
--keytab_file=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786517687384214-19665-0/krb5kdc/kudu_127.19.52.67.keytab
--principal=kudu/127.19.52.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.52.84:44333
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786517687384214-19665-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786517687384214-19665-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786517687384214-19665-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 06:54:52.358100 20695 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:54:52.358381 20695 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:54:52.358428 20695 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 06:54:52.362624 20695 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 06:54:52.362694 20695 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 06:54:52.362757 20695 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.52.67
I20260812 06:54:52.368788 20695 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.52.84:44333
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786517687384214-19665-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786517687384214-19665-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786517687384214-19665-0/krb5kdc/kudu_127.19.52.67.keytab
--principal=kudu/127.19.52.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.52.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786517687384214-19665-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.19.52.67
--webserver_port=0
--tserver_master_addrs=127.19.52.126:33709
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20695
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.52.67
--log_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786517687384214-19665-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 06:15:54 UTC on ccycloud.quasar-rnfzce.root.comops.site
build id 4581989
I20260812 06:54:52.370014 20695 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:54:52.371156 20695 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 06:54:52.381526 20701 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:54:52.381526 20705 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:54:52.381526 20703 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:54:52.381908 20695 server_base.cc:1034] running on GCE node
I20260812 06:54:52.382344 20695 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:54:52.383057 20695 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:54:52.384292 20695 hybrid_clock.cc:648] HybridClock initialized: now 1786517692384259 us; error 48 us; skew 500 ppm
Aug 12 06:54:52 dist-test-slave-65mx krb5kdc[20255](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517692, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.52.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 06:54:52.387373 20695 init.cc:377] Logged in from keytab as kudu/127.19.52.67@KRBTEST.COM (short username kudu)
I20260812 06:54:52.388335 20695 webserver.cc:460] Webserver started at http://127.19.52.67:42647/ using document root <none> and password file <none>
I20260812 06:54:52.388983 20695 fs_manager.cc:359] Metadata directory not provided
I20260812 06:54:52.389053 20695 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:54:52.389245 20695 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 06:54:52.391383 20695 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786517687384214-19665-0/minicluster-data/ts-2/data/instance:
uuid: "1e464ffec54649d3af8fe62c2fe6076a"
format_stamp: "Formatted at 2026-08-12 06:54:52 on dist-test-slave-65mx"
server_key: "d75627f54a86d4b3bd7cfaee7717c366"
server_key_iv: "c0676716373a4c033fb60aad31e87616"
server_key_version: "encryptionkey@0"
I20260812 06:54:52.391937 20695 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786517687384214-19665-0/minicluster-data/ts-2/wal/instance:
uuid: "1e464ffec54649d3af8fe62c2fe6076a"
format_stamp: "Formatted at 2026-08-12 06:54:52 on dist-test-slave-65mx"
server_key: "d75627f54a86d4b3bd7cfaee7717c366"
server_key_iv: "c0676716373a4c033fb60aad31e87616"
server_key_version: "encryptionkey@0"
I20260812 06:54:52.396147 20695 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260812 06:54:52.398854 20712 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 06:54:52.400393 20695 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260812 06:54:52.400499 20695 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786517687384214-19665-0/minicluster-data/ts-2/data,/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786517687384214-19665-0/minicluster-data/ts-2/wal
uuid: "1e464ffec54649d3af8fe62c2fe6076a"
format_stamp: "Formatted at 2026-08-12 06:54:52 on dist-test-slave-65mx"
server_key: "d75627f54a86d4b3bd7cfaee7717c366"
server_key_iv: "c0676716373a4c033fb60aad31e87616"
server_key_version: "encryptionkey@0"
I20260812 06:54:52.400660 20695 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786517687384214-19665-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786517687384214-19665-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786517687384214-19665-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:54:52.426573 20695 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 06:54:52.430713 20695 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 06:54:52.430948 20695 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:54:52.431669 20695 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 06:54:52.432619 20695 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 06:54:52.432670 20695 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 06:54:52.432741 20695 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 06:54:52.432775 20695 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 06:54:52.442502 20695 rpc_server.cc:307] RPC server started. Bound to: 127.19.52.67:35041
I20260812 06:54:52.442664 20838 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.52.67:35041 every 8 connection(s)
I20260812 06:54:52.443552 20695 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786517687384214-19665-0/minicluster-data/ts-2/data/info.pb
I20260812 06:54:52.446257 19665 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu as pid 20695
I20260812 06:54:52.446388 19665 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786517687384214-19665-0/minicluster-data/ts-2/wal/instance
I20260812 06:54:52.446723 19665 external_mini_cluster.cc:1428] Setting key fd7c0ddf60acfe999756d0c45d3de94c
Aug 12 06:54:52 dist-test-slave-65mx krb5kdc[20255](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517692, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.52.67@KRBTEST.COM for kudu/127.19.52.126@KRBTEST.COM
I20260812 06:54:52.458109 20361 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:54:52.445513 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.52.67:46825 (local address 127.19.52.126:33709)
0812 06:54:52.445645 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0812 06:54:52.445648 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 06:54:52.446677 (+ 1029us) server_negotiation.cc:407] Connection header received
0812 06:54:52.448432 (+ 1755us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:54:52.448435 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:54:52.448491 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:54:52.448595 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:54:52.449902 (+ 1307us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:54:52.450460 (+ 558us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:54:52.451575 (+ 1115us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:54:52.451795 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:54:52.451877 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:54:52.454398 (+ 2521us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:54:52.454401 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:54:52.454403 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:54:52.454430 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:54:52.455953 (+ 1523us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:54:52.456436 (+ 483us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:54:52.456441 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:54:52.456443 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:54:52.456534 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:54:52.456940 (+ 406us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:54:52.456944 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:54:52.456946 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:54:52.457153 (+ 207us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:54:52.457267 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0812 06:54:52.457789 (+ 522us) server_negotiation.cc:299] Negotiation successful
0812 06:54:52.457922 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260812 06:54:52.459131 20841 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:54:52.445868 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.126:33709 (local address 127.19.52.67:46825)
0812 06:54:52.446487 (+ 619us) negotiation.cc:107] Waiting for socket to connect
0812 06:54:52.446542 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0812 06:54:52.448178 (+ 1636us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:54:52.448787 (+ 609us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:54:52.448794 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:54:52.449263 (+ 469us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:54:52.449766 (+ 503us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:54:52.449780 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:54:52.450566 (+ 786us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:54:52.450569 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:54:52.451456 (+ 887us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:54:52.451465 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:54:52.451933 (+ 468us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:54:52.451935 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:54:52.452077 (+ 142us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:54:52.452750 (+ 673us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 06:54:52.452767 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:54:52.454237 (+ 1470us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 06:54:52.456057 (+ 1820us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:54:52.456062 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:54:52.456073 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:54:52.456322 (+ 249us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:54:52.456669 (+ 347us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:54:52.456672 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:54:52.456674 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:54:52.456823 (+ 149us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:54:52.457266 (+ 443us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 06:54:52.457275 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 06:54:52.457532 (+ 257us) client_negotiation.cc:769] Sending connection context
0812 06:54:52.457743 (+ 211us) client_negotiation.cc:240] Negotiation successful
0812 06:54:52.458124 (+ 381us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":370,"thread_start_us":172,"threads_started":1}
I20260812 06:54:52.460410 20839 heartbeater.cc:347] Connected to a master server at 127.19.52.126:33709
I20260812 06:54:52.460722 20839 heartbeater.cc:464] Registering TS with master...
I20260812 06:54:52.461323 20839 heartbeater.cc:511] Master 127.19.52.126:33709 requested a full tablet report, sending...
I20260812 06:54:52.462352 20306 ts_manager.cc:194] Registered new tserver with Master: 1e464ffec54649d3af8fe62c2fe6076a (127.19.52.67:35041)
I20260812 06:54:52.463088 20306 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.52.67@KRBTEST.COM'} at 127.19.52.67:46825
I20260812 06:54:52.473271 19665 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 06:54:52.485581 20361 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:54:52.477676 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40700 (local address 127.19.52.126:33709)
0812 06:54:52.477815 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0812 06:54:52.477824 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 06:54:52.478123 (+ 299us) server_negotiation.cc:407] Connection header received
0812 06:54:52.478246 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:54:52.478249 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:54:52.478307 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:54:52.478425 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:54:52.479343 (+ 918us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:54:52.479885 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:54:52.480746 (+ 861us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:54:52.480960 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:54:52.481041 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:54:52.482418 (+ 1377us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:54:52.482421 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:54:52.482424 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:54:52.482448 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:54:52.483972 (+ 1524us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:54:52.484402 (+ 430us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:54:52.484404 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:54:52.484405 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:54:52.484483 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:54:52.484792 (+ 309us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:54:52.484794 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:54:52.484795 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:54:52.485023 (+ 228us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:54:52.485153 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0812 06:54:52.485359 (+ 206us) server_negotiation.cc:299] Negotiation successful
0812 06:54:52.485456 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260812 06:54:52.491762 20306 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40700:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 06:54:52.496003 20306 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 06:54:52.513078 20851 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:54:52.507205 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.66:37985 (local address 127.0.0.1:33232)
0812 06:54:52.508610 (+ 1405us) negotiation.cc:107] Waiting for socket to connect
0812 06:54:52.508626 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0812 06:54:52.508726 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:54:52.509133 (+ 407us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:54:52.509136 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:54:52.509154 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 06:54:52.509272 (+ 118us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:54:52.509281 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:54:52.510404 (+ 1123us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:54:52.510407 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:54:52.511597 (+ 1190us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:54:52.511608 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:54:52.512448 (+ 840us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:54:52.512451 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:54:52.512616 (+ 165us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:54:52.512669 (+ 53us) client_negotiation.cc:769] Sending connection context
0812 06:54:52.512779 (+ 110us) client_negotiation.cc:240] Negotiation successful
0812 06:54:52.512903 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1294,"mutex_wait_us":8,"thread_start_us":112,"threads_started":1}
I20260812 06:54:52.513698 20855 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:54:52.507592 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33232 (local address 127.19.52.66:37985)
0812 06:54:52.508039 (+ 447us) server_negotiation.cc:206] Beginning negotiation
0812 06:54:52.508059 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0812 06:54:52.508725 (+ 666us) server_negotiation.cc:407] Connection header received
0812 06:54:52.508835 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:54:52.508847 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:54:52.508993 (+ 146us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:54:52.509147 (+ 154us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 06:54:52.509372 (+ 225us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:54:52.510266 (+ 894us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:54:52.511733 (+ 1467us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:54:52.512315 (+ 582us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:54:52.512409 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:54:52.512684 (+ 275us) server_negotiation.cc:1035] Waiting for connection context
0812 06:54:52.513415 (+ 731us) server_negotiation.cc:299] Negotiation successful
0812 06:54:52.513523 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":353,"thread_start_us":134,"threads_started":1}
I20260812 06:54:52.514043 20850 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:54:52.506482 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.65:37203 (local address 127.0.0.1:42868)
0812 06:54:52.507004 (+ 522us) negotiation.cc:107] Waiting for socket to connect
0812 06:54:52.507070 (+ 66us) client_negotiation.cc:174] Beginning negotiation
0812 06:54:52.507205 (+ 135us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:54:52.508360 (+ 1155us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:54:52.508364 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:54:52.508384 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 06:54:52.508514 (+ 130us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:54:52.508527 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:54:52.510092 (+ 1565us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:54:52.510095 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:54:52.511597 (+ 1502us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:54:52.511608 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:54:52.513648 (+ 2040us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:54:52.513651 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:54:52.513757 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:54:52.513789 (+ 32us) client_negotiation.cc:769] Sending connection context
0812 06:54:52.513846 (+ 57us) client_negotiation.cc:240] Negotiation successful
0812 06:54:52.513918 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":361,"mutex_wait_us":26,"thread_start_us":104,"threads_started":1}
I20260812 06:54:52.514235 20853 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:54:52.506688 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.67:35041 (local address 127.0.0.1:44036)
0812 06:54:52.507567 (+ 879us) negotiation.cc:107] Waiting for socket to connect
0812 06:54:52.507582 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 06:54:52.507674 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:54:52.508293 (+ 619us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:54:52.508299 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:54:52.508323 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 06:54:52.508493 (+ 170us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:54:52.508501 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:54:52.510082 (+ 1581us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:54:52.510085 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:54:52.512445 (+ 2360us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:54:52.512453 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:54:52.513789 (+ 1336us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:54:52.513795 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:54:52.513913 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:54:52.513935 (+ 22us) client_negotiation.cc:769] Sending connection context
0812 06:54:52.514003 (+ 68us) client_negotiation.cc:240] Negotiation successful
0812 06:54:52.514073 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":804,"mutex_wait_us":2,"thread_start_us":108,"threads_started":1}
I20260812 06:54:52.514807 20854 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:54:52.507268 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44036 (local address 127.19.52.67:35041)
0812 06:54:52.507792 (+ 524us) server_negotiation.cc:206] Beginning negotiation
0812 06:54:52.507807 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0812 06:54:52.507848 (+ 41us) server_negotiation.cc:407] Connection header received
0812 06:54:52.507932 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:54:52.507937 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:54:52.508124 (+ 187us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:54:52.508299 (+ 175us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 06:54:52.508656 (+ 357us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:54:52.509916 (+ 1260us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:54:52.512615 (+ 2699us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:54:52.513438 (+ 823us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:54:52.513579 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:54:52.513729 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0812 06:54:52.514527 (+ 798us) server_negotiation.cc:299] Negotiation successful
0812 06:54:52.514641 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":422,"thread_start_us":210,"threads_started":1}
I20260812 06:54:52.515717 20852 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:54:52.506665 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42868 (local address 127.19.52.65:37203)
0812 06:54:52.507931 (+ 1266us) server_negotiation.cc:206] Beginning negotiation
0812 06:54:52.507939 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 06:54:52.507963 (+ 24us) server_negotiation.cc:407] Connection header received
0812 06:54:52.508043 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:54:52.508050 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:54:52.508219 (+ 169us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:54:52.508377 (+ 158us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 06:54:52.508705 (+ 328us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:54:52.509949 (+ 1244us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:54:52.512650 (+ 2701us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:54:52.513494 (+ 844us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:54:52.514314 (+ 820us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:54:52.514452 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0812 06:54:52.514510 (+ 58us) server_negotiation.cc:299] Negotiation successful
0812 06:54:52.515605 (+ 1095us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1147,"thread_start_us":158,"threads_started":1}
I20260812 06:54:52.517343 20767 tablet_service.cc:1467] Processing CreateTablet for tablet f177fe529afb44e6841e29fca9783e43 (DEFAULT_TABLE table=test-table [id=00af4166349648dd8f19df9a3e9dbdd5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 06:54:52.517175 20610 tablet_service.cc:1467] Processing CreateTablet for tablet f177fe529afb44e6841e29fca9783e43 (DEFAULT_TABLE table=test-table [id=00af4166349648dd8f19df9a3e9dbdd5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 06:54:52.517889 20453 tablet_service.cc:1467] Processing CreateTablet for tablet f177fe529afb44e6841e29fca9783e43 (DEFAULT_TABLE table=test-table [id=00af4166349648dd8f19df9a3e9dbdd5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 06:54:52.518380 20610 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f177fe529afb44e6841e29fca9783e43. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 06:54:52.518937 20453 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f177fe529afb44e6841e29fca9783e43. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 06:54:52.519601 20767 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f177fe529afb44e6841e29fca9783e43. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 06:54:52.525632 20856 tablet_bootstrap.cc:492] T f177fe529afb44e6841e29fca9783e43 P ef158262d7574ab998d2ead0bf1f300d: Bootstrap starting.
I20260812 06:54:52.526382 20857 tablet_bootstrap.cc:492] T f177fe529afb44e6841e29fca9783e43 P ff835b4f96b849cf934c9ea972c47293: Bootstrap starting.
I20260812 06:54:52.527860 20858 tablet_bootstrap.cc:492] T f177fe529afb44e6841e29fca9783e43 P 1e464ffec54649d3af8fe62c2fe6076a: Bootstrap starting.
I20260812 06:54:52.529749 20856 tablet_bootstrap.cc:654] T f177fe529afb44e6841e29fca9783e43 P ef158262d7574ab998d2ead0bf1f300d: Neither blocks nor log segments found. Creating new log.
I20260812 06:54:52.530524 20857 tablet_bootstrap.cc:654] T f177fe529afb44e6841e29fca9783e43 P ff835b4f96b849cf934c9ea972c47293: Neither blocks nor log segments found. Creating new log.
I20260812 06:54:52.530743 20858 tablet_bootstrap.cc:654] T f177fe529afb44e6841e29fca9783e43 P 1e464ffec54649d3af8fe62c2fe6076a: Neither blocks nor log segments found. Creating new log.
I20260812 06:54:52.531168 20856 log.cc:824] T f177fe529afb44e6841e29fca9783e43 P ef158262d7574ab998d2ead0bf1f300d: Log is configured to *not* fsync() on all Append() calls
I20260812 06:54:52.532069 20857 log.cc:824] T f177fe529afb44e6841e29fca9783e43 P ff835b4f96b849cf934c9ea972c47293: Log is configured to *not* fsync() on all Append() calls
I20260812 06:54:52.532792 20858 log.cc:824] T f177fe529afb44e6841e29fca9783e43 P 1e464ffec54649d3af8fe62c2fe6076a: Log is configured to *not* fsync() on all Append() calls
I20260812 06:54:52.533722 20856 tablet_bootstrap.cc:492] T f177fe529afb44e6841e29fca9783e43 P ef158262d7574ab998d2ead0bf1f300d: No bootstrap required, opened a new log
I20260812 06:54:52.533931 20856 ts_tablet_manager.cc:1397] T f177fe529afb44e6841e29fca9783e43 P ef158262d7574ab998d2ead0bf1f300d: Time spent bootstrapping tablet: real 0.008s user 0.006s sys 0.000s
I20260812 06:54:52.534348 20857 tablet_bootstrap.cc:492] T f177fe529afb44e6841e29fca9783e43 P ff835b4f96b849cf934c9ea972c47293: No bootstrap required, opened a new log
I20260812 06:54:52.534569 20857 ts_tablet_manager.cc:1397] T f177fe529afb44e6841e29fca9783e43 P ff835b4f96b849cf934c9ea972c47293: Time spent bootstrapping tablet: real 0.008s user 0.003s sys 0.003s
I20260812 06:54:52.535104 20858 tablet_bootstrap.cc:492] T f177fe529afb44e6841e29fca9783e43 P 1e464ffec54649d3af8fe62c2fe6076a: No bootstrap required, opened a new log
I20260812 06:54:52.535288 20858 ts_tablet_manager.cc:1397] T f177fe529afb44e6841e29fca9783e43 P 1e464ffec54649d3af8fe62c2fe6076a: Time spent bootstrapping tablet: real 0.008s user 0.003s sys 0.003s
I20260812 06:54:52.539562 20856 raft_consensus.cc:348] T f177fe529afb44e6841e29fca9783e43 P ef158262d7574ab998d2ead0bf1f300d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef158262d7574ab998d2ead0bf1f300d" member_type: VOTER last_known_addr { host: "127.19.52.65" port: 37203 } } peers { permanent_uuid: "1e464ffec54649d3af8fe62c2fe6076a" member_type: VOTER last_known_addr { host: "127.19.52.67" port: 35041 } } peers { permanent_uuid: "ff835b4f96b849cf934c9ea972c47293" member_type: VOTER last_known_addr { host: "127.19.52.66" port: 37985 } }
I20260812 06:54:52.539839 20856 raft_consensus.cc:374] T f177fe529afb44e6841e29fca9783e43 P ef158262d7574ab998d2ead0bf1f300d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 06:54:52.539894 20856 raft_consensus.cc:729] T f177fe529afb44e6841e29fca9783e43 P ef158262d7574ab998d2ead0bf1f300d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ef158262d7574ab998d2ead0bf1f300d, State: Initialized, Role: FOLLOWER
I20260812 06:54:52.540092 20857 raft_consensus.cc:348] T f177fe529afb44e6841e29fca9783e43 P ff835b4f96b849cf934c9ea972c47293 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef158262d7574ab998d2ead0bf1f300d" member_type: VOTER last_known_addr { host: "127.19.52.65" port: 37203 } } peers { permanent_uuid: "1e464ffec54649d3af8fe62c2fe6076a" member_type: VOTER last_known_addr { host: "127.19.52.67" port: 35041 } } peers { permanent_uuid: "ff835b4f96b849cf934c9ea972c47293" member_type: VOTER last_known_addr { host: "127.19.52.66" port: 37985 } }
I20260812 06:54:52.540423 20857 raft_consensus.cc:374] T f177fe529afb44e6841e29fca9783e43 P ff835b4f96b849cf934c9ea972c47293 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 06:54:52.540362 20856 consensus_queue.cc:260] T f177fe529afb44e6841e29fca9783e43 P ef158262d7574ab998d2ead0bf1f300d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef158262d7574ab998d2ead0bf1f300d" member_type: VOTER last_known_addr { host: "127.19.52.65" port: 37203 } } peers { permanent_uuid: "1e464ffec54649d3af8fe62c2fe6076a" member_type: VOTER last_known_addr { host: "127.19.52.67" port: 35041 } } peers { permanent_uuid: "ff835b4f96b849cf934c9ea972c47293" member_type: VOTER last_known_addr { host: "127.19.52.66" port: 37985 } }
I20260812 06:54:52.540514 20857 raft_consensus.cc:729] T f177fe529afb44e6841e29fca9783e43 P ff835b4f96b849cf934c9ea972c47293 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ff835b4f96b849cf934c9ea972c47293, State: Initialized, Role: FOLLOWER
I20260812 06:54:52.540525 20858 raft_consensus.cc:348] T f177fe529afb44e6841e29fca9783e43 P 1e464ffec54649d3af8fe62c2fe6076a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef158262d7574ab998d2ead0bf1f300d" member_type: VOTER last_known_addr { host: "127.19.52.65" port: 37203 } } peers { permanent_uuid: "1e464ffec54649d3af8fe62c2fe6076a" member_type: VOTER last_known_addr { host: "127.19.52.67" port: 35041 } } peers { permanent_uuid: "ff835b4f96b849cf934c9ea972c47293" member_type: VOTER last_known_addr { host: "127.19.52.66" port: 37985 } }
I20260812 06:54:52.540879 20858 raft_consensus.cc:374] T f177fe529afb44e6841e29fca9783e43 P 1e464ffec54649d3af8fe62c2fe6076a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 06:54:52.540943 20858 raft_consensus.cc:729] T f177fe529afb44e6841e29fca9783e43 P 1e464ffec54649d3af8fe62c2fe6076a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1e464ffec54649d3af8fe62c2fe6076a, State: Initialized, Role: FOLLOWER
I20260812 06:54:52.541054 20857 consensus_queue.cc:260] T f177fe529afb44e6841e29fca9783e43 P ff835b4f96b849cf934c9ea972c47293 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef158262d7574ab998d2ead0bf1f300d" member_type: VOTER last_known_addr { host: "127.19.52.65" port: 37203 } } peers { permanent_uuid: "1e464ffec54649d3af8fe62c2fe6076a" member_type: VOTER last_known_addr { host: "127.19.52.67" port: 35041 } } peers { permanent_uuid: "ff835b4f96b849cf934c9ea972c47293" member_type: VOTER last_known_addr { host: "127.19.52.66" port: 37985 } }
I20260812 06:54:52.541496 20529 heartbeater.cc:503] Master 127.19.52.126:33709 was elected leader, sending a full tablet report...
I20260812 06:54:52.541450 20858 consensus_queue.cc:260] T f177fe529afb44e6841e29fca9783e43 P 1e464ffec54649d3af8fe62c2fe6076a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef158262d7574ab998d2ead0bf1f300d" member_type: VOTER last_known_addr { host: "127.19.52.65" port: 37203 } } peers { permanent_uuid: "1e464ffec54649d3af8fe62c2fe6076a" member_type: VOTER last_known_addr { host: "127.19.52.67" port: 35041 } } peers { permanent_uuid: "ff835b4f96b849cf934c9ea972c47293" member_type: VOTER last_known_addr { host: "127.19.52.66" port: 37985 } }
I20260812 06:54:52.541661 20856 ts_tablet_manager.cc:1428] T f177fe529afb44e6841e29fca9783e43 P ef158262d7574ab998d2ead0bf1f300d: Time spent starting tablet: real 0.008s user 0.009s sys 0.000s
I20260812 06:54:52.542107 20857 ts_tablet_manager.cc:1428] T f177fe529afb44e6841e29fca9783e43 P ff835b4f96b849cf934c9ea972c47293: Time spent starting tablet: real 0.007s user 0.008s sys 0.000s
I20260812 06:54:52.542295 20690 heartbeater.cc:503] Master 127.19.52.126:33709 was elected leader, sending a full tablet report...
I20260812 06:54:52.542382 20858 ts_tablet_manager.cc:1428] T f177fe529afb44e6841e29fca9783e43 P 1e464ffec54649d3af8fe62c2fe6076a: Time spent starting tablet: real 0.007s user 0.004s sys 0.004s
I20260812 06:54:52.542727 20839 heartbeater.cc:503] Master 127.19.52.126:33709 was elected leader, sending a full tablet report...
I20260812 06:54:52.568863 20864 raft_consensus.cc:482] T f177fe529afb44e6841e29fca9783e43 P 1e464ffec54649d3af8fe62c2fe6076a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 06:54:52.568815 20862 raft_consensus.cc:482] T f177fe529afb44e6841e29fca9783e43 P ef158262d7574ab998d2ead0bf1f300d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 06:54:52.569149 20862 raft_consensus.cc:504] T f177fe529afb44e6841e29fca9783e43 P ef158262d7574ab998d2ead0bf1f300d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef158262d7574ab998d2ead0bf1f300d" member_type: VOTER last_known_addr { host: "127.19.52.65" port: 37203 } } peers { permanent_uuid: "1e464ffec54649d3af8fe62c2fe6076a" member_type: VOTER last_known_addr { host: "127.19.52.67" port: 35041 } } peers { permanent_uuid: "ff835b4f96b849cf934c9ea972c47293" member_type: VOTER last_known_addr { host: "127.19.52.66" port: 37985 } }
I20260812 06:54:52.569123 20864 raft_consensus.cc:504] T f177fe529afb44e6841e29fca9783e43 P 1e464ffec54649d3af8fe62c2fe6076a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef158262d7574ab998d2ead0bf1f300d" member_type: VOTER last_known_addr { host: "127.19.52.65" port: 37203 } } peers { permanent_uuid: "1e464ffec54649d3af8fe62c2fe6076a" member_type: VOTER last_known_addr { host: "127.19.52.67" port: 35041 } } peers { permanent_uuid: "ff835b4f96b849cf934c9ea972c47293" member_type: VOTER last_known_addr { host: "127.19.52.66" port: 37985 } }
I20260812 06:54:52.570430 20862 leader_election.cc:288] T f177fe529afb44e6841e29fca9783e43 P ef158262d7574ab998d2ead0bf1f300d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1e464ffec54649d3af8fe62c2fe6076a (127.19.52.67:35041), ff835b4f96b849cf934c9ea972c47293 (127.19.52.66:37985)
I20260812 06:54:52.570374 20864 leader_election.cc:288] T f177fe529afb44e6841e29fca9783e43 P 1e464ffec54649d3af8fe62c2fe6076a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ef158262d7574ab998d2ead0bf1f300d (127.19.52.65:37203), ff835b4f96b849cf934c9ea972c47293 (127.19.52.66:37985)
I20260812 06:54:52.576038 20841 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:54:52.570615 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.65:37203 (local address 127.19.52.67:34597)
0812 06:54:52.570849 (+ 234us) negotiation.cc:107] Waiting for socket to connect
0812 06:54:52.570864 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 06:54:52.570958 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:54:52.571912 (+ 954us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:54:52.571915 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:54:52.571934 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 06:54:52.572163 (+ 229us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:54:52.572172 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:54:52.573274 (+ 1102us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:54:52.573277 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:54:52.574569 (+ 1292us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:54:52.574580 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:54:52.575507 (+ 927us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:54:52.575513 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:54:52.575625 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:54:52.575727 (+ 102us) client_negotiation.cc:769] Sending connection context
0812 06:54:52.575795 (+ 68us) client_negotiation.cc:240] Negotiation successful
0812 06:54:52.575880 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":146,"mutex_wait_us":62}
I20260812 06:54:52.576347 20855 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:54:52.570933 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.52.65:33791 (local address 127.19.52.66:37985)
0812 06:54:52.571071 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0812 06:54:52.571076 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 06:54:52.571863 (+ 787us) server_negotiation.cc:407] Connection header received
0812 06:54:52.572030 (+ 167us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:54:52.572035 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:54:52.572115 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:54:52.572166 (+ 51us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 06:54:52.572610 (+ 444us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:54:52.573307 (+ 697us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:54:52.575012 (+ 1705us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:54:52.575643 (+ 631us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:54:52.575761 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:54:52.575819 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0812 06:54:52.576121 (+ 302us) server_negotiation.cc:299] Negotiation successful
0812 06:54:52.576201 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":23,"server-negotiator.queue_time_us":56}
I20260812 06:54:52.576351 20854 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:54:52.571114 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.52.65:48069 (local address 127.19.52.67:35041)
0812 06:54:52.571241 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0812 06:54:52.571248 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 06:54:52.571540 (+ 292us) server_negotiation.cc:407] Connection header received
0812 06:54:52.571792 (+ 252us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:54:52.571795 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:54:52.571856 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:54:52.571911 (+ 55us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 06:54:52.572417 (+ 506us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:54:52.573233 (+ 816us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:54:52.575014 (+ 1781us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:54:52.575686 (+ 672us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:54:52.575795 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:54:52.575846 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0812 06:54:52.576182 (+ 336us) server_negotiation.cc:299] Negotiation successful
0812 06:54:52.576235 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":4,"server-negotiator.queue_time_us":47,"spinlock_wait_cycles":8832}
I20260812 06:54:52.576359 20867 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:54:52.570867 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.66:37985 (local address 127.19.52.67:46973)
0812 06:54:52.571448 (+ 581us) negotiation.cc:107] Waiting for socket to connect
0812 06:54:52.571462 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 06:54:52.571560 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:54:52.572104 (+ 544us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:54:52.572107 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:54:52.572121 (+ 14us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 06:54:52.572385 (+ 264us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:54:52.572397 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:54:52.573622 (+ 1225us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:54:52.573626 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:54:52.574833 (+ 1207us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:54:52.574844 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:54:52.575973 (+ 1129us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:54:52.575977 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:54:52.576066 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:54:52.576088 (+ 22us) client_negotiation.cc:769] Sending connection context
0812 06:54:52.576138 (+ 50us) client_negotiation.cc:240] Negotiation successful
0812 06:54:52.576211 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":514,"thread_start_us":132,"threads_started":1}
I20260812 06:54:52.576587 20852 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:54:52.571466 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.52.67:34597 (local address 127.19.52.65:37203)
0812 06:54:52.571572 (+ 106us) server_negotiation.cc:206] Beginning negotiation
0812 06:54:52.571576 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 06:54:52.571597 (+ 21us) server_negotiation.cc:407] Connection header received
0812 06:54:52.571649 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:54:52.571655 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:54:52.571706 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:54:52.571778 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 06:54:52.572295 (+ 517us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:54:52.573152 (+ 857us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:54:52.574710 (+ 1558us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:54:52.575284 (+ 574us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:54:52.575395 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:54:52.575446 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0812 06:54:52.576345 (+ 899us) server_negotiation.cc:299] Negotiation successful
0812 06:54:52.576429 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":28,"server-negotiator.queue_time_us":47}
I20260812 06:54:52.576982 20866 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:54:52.570678 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.67:35041 (local address 127.19.52.65:48069)
0812 06:54:52.571451 (+ 773us) negotiation.cc:107] Waiting for socket to connect
0812 06:54:52.571467 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0812 06:54:52.571640 (+ 173us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:54:52.572097 (+ 457us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:54:52.572099 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:54:52.572116 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 06:54:52.572253 (+ 137us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:54:52.572260 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:54:52.573378 (+ 1118us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:54:52.573381 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:54:52.574887 (+ 1506us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:54:52.574899 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:54:52.575911 (+ 1012us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:54:52.575916 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:54:52.576027 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:54:52.576065 (+ 38us) client_negotiation.cc:769] Sending connection context
0812 06:54:52.576682 (+ 617us) client_negotiation.cc:240] Negotiation successful
0812 06:54:52.576757 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":655,"mutex_wait_us":86,"thread_start_us":178,"threads_started":1}
I20260812 06:54:52.576996 20868 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:54:52.570678 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.66:37985 (local address 127.19.52.65:33791)
0812 06:54:52.571787 (+ 1109us) negotiation.cc:107] Waiting for socket to connect
0812 06:54:52.571801 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 06:54:52.571921 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:54:52.572312 (+ 391us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:54:52.572315 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:54:52.572334 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 06:54:52.572460 (+ 126us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:54:52.572466 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:54:52.573437 (+ 971us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:54:52.573440 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:54:52.574887 (+ 1447us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:54:52.574897 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:54:52.575880 (+ 983us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:54:52.575884 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:54:52.575979 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:54:52.576000 (+ 21us) client_negotiation.cc:769] Sending connection context
0812 06:54:52.576482 (+ 482us) client_negotiation.cc:240] Negotiation successful
0812 06:54:52.576575 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1042,"mutex_wait_us":446,"thread_start_us":118,"threads_started":1}
I20260812 06:54:52.577327 20476 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f177fe529afb44e6841e29fca9783e43" candidate_uuid: "1e464ffec54649d3af8fe62c2fe6076a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ef158262d7574ab998d2ead0bf1f300d" is_pre_election: true
I20260812 06:54:52.577636 20636 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f177fe529afb44e6841e29fca9783e43" candidate_uuid: "ef158262d7574ab998d2ead0bf1f300d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ff835b4f96b849cf934c9ea972c47293" is_pre_election: true
I20260812 06:54:52.577770 20476 raft_consensus.cc:2454] T f177fe529afb44e6841e29fca9783e43 P ef158262d7574ab998d2ead0bf1f300d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1e464ffec54649d3af8fe62c2fe6076a in term 0.
I20260812 06:54:52.577915 20869 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:54:52.571186 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.52.67:46973 (local address 127.19.52.66:37985)
0812 06:54:52.571820 (+ 634us) server_negotiation.cc:206] Beginning negotiation
0812 06:54:52.571861 (+ 41us) server_negotiation.cc:399] Waiting for connection header
0812 06:54:52.571875 (+ 14us) server_negotiation.cc:407] Connection header received
0812 06:54:52.571928 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:54:52.571930 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:54:52.571974 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:54:52.572494 (+ 520us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 06:54:52.572633 (+ 139us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:54:52.573491 (+ 858us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:54:52.574976 (+ 1485us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:54:52.575847 (+ 871us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:54:52.577334 (+ 1487us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:54:52.577392 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0812 06:54:52.577427 (+ 35us) server_negotiation.cc:299] Negotiation successful
0812 06:54:52.577489 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":195,"server-negotiator.queue_time_us":568,"thread_start_us":204,"threads_started":1}
I20260812 06:54:52.577975 20636 raft_consensus.cc:2454] T f177fe529afb44e6841e29fca9783e43 P ff835b4f96b849cf934c9ea972c47293 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef158262d7574ab998d2ead0bf1f300d in term 0.
I20260812 06:54:52.578394 20715 leader_election.cc:302] T f177fe529afb44e6841e29fca9783e43 P 1e464ffec54649d3af8fe62c2fe6076a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1e464ffec54649d3af8fe62c2fe6076a, ef158262d7574ab998d2ead0bf1f300d; no voters:
I20260812 06:54:52.578473 20787 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f177fe529afb44e6841e29fca9783e43" candidate_uuid: "ef158262d7574ab998d2ead0bf1f300d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1e464ffec54649d3af8fe62c2fe6076a" is_pre_election: true
I20260812 06:54:52.578497 20404 leader_election.cc:302] T f177fe529afb44e6841e29fca9783e43 P ef158262d7574ab998d2ead0bf1f300d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ef158262d7574ab998d2ead0bf1f300d, ff835b4f96b849cf934c9ea972c47293; no voters:
I20260812 06:54:52.578652 20787 raft_consensus.cc:2454] T f177fe529afb44e6841e29fca9783e43 P 1e464ffec54649d3af8fe62c2fe6076a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef158262d7574ab998d2ead0bf1f300d in term 0.
I20260812 06:54:52.578737 20864 raft_consensus.cc:2781] T f177fe529afb44e6841e29fca9783e43 P 1e464ffec54649d3af8fe62c2fe6076a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 06:54:52.578809 20864 raft_consensus.cc:482] T f177fe529afb44e6841e29fca9783e43 P 1e464ffec54649d3af8fe62c2fe6076a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 06:54:52.578773 20636 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f177fe529afb44e6841e29fca9783e43" candidate_uuid: "1e464ffec54649d3af8fe62c2fe6076a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ff835b4f96b849cf934c9ea972c47293" is_pre_election: true
I20260812 06:54:52.578856 20864 raft_consensus.cc:3037] T f177fe529afb44e6841e29fca9783e43 P 1e464ffec54649d3af8fe62c2fe6076a [term 0 FOLLOWER]: Advancing to term 1
I20260812 06:54:52.578896 20636 raft_consensus.cc:2454] T f177fe529afb44e6841e29fca9783e43 P ff835b4f96b849cf934c9ea972c47293 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1e464ffec54649d3af8fe62c2fe6076a in term 0.
I20260812 06:54:52.578979 20862 raft_consensus.cc:2781] T f177fe529afb44e6841e29fca9783e43 P ef158262d7574ab998d2ead0bf1f300d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 06:54:52.579061 20862 raft_consensus.cc:482] T f177fe529afb44e6841e29fca9783e43 P ef158262d7574ab998d2ead0bf1f300d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 06:54:52.579121 20862 raft_consensus.cc:3037] T f177fe529afb44e6841e29fca9783e43 P ef158262d7574ab998d2ead0bf1f300d [term 0 FOLLOWER]: Advancing to term 1
I20260812 06:54:52.580330 20864 raft_consensus.cc:504] T f177fe529afb44e6841e29fca9783e43 P 1e464ffec54649d3af8fe62c2fe6076a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef158262d7574ab998d2ead0bf1f300d" member_type: VOTER last_known_addr { host: "127.19.52.65" port: 37203 } } peers { permanent_uuid: "1e464ffec54649d3af8fe62c2fe6076a" member_type: VOTER last_known_addr { host: "127.19.52.67" port: 35041 } } peers { permanent_uuid: "ff835b4f96b849cf934c9ea972c47293" member_type: VOTER last_known_addr { host: "127.19.52.66" port: 37985 } }
I20260812 06:54:52.580370 20862 raft_consensus.cc:504] T f177fe529afb44e6841e29fca9783e43 P ef158262d7574ab998d2ead0bf1f300d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef158262d7574ab998d2ead0bf1f300d" member_type: VOTER last_known_addr { host: "127.19.52.65" port: 37203 } } peers { permanent_uuid: "1e464ffec54649d3af8fe62c2fe6076a" member_type: VOTER last_known_addr { host: "127.19.52.67" port: 35041 } } peers { permanent_uuid: "ff835b4f96b849cf934c9ea972c47293" member_type: VOTER last_known_addr { host: "127.19.52.66" port: 37985 } }
I20260812 06:54:52.580744 20864 leader_election.cc:288] T f177fe529afb44e6841e29fca9783e43 P 1e464ffec54649d3af8fe62c2fe6076a [CANDIDATE]: Term 1 election: Requested vote from peers ef158262d7574ab998d2ead0bf1f300d (127.19.52.65:37203), ff835b4f96b849cf934c9ea972c47293 (127.19.52.66:37985)
I20260812 06:54:52.580751 20862 leader_election.cc:288] T f177fe529afb44e6841e29fca9783e43 P ef158262d7574ab998d2ead0bf1f300d [CANDIDATE]: Term 1 election: Requested vote from peers 1e464ffec54649d3af8fe62c2fe6076a (127.19.52.67:35041), ff835b4f96b849cf934c9ea972c47293 (127.19.52.66:37985)
I20260812 06:54:52.581182 20636 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f177fe529afb44e6841e29fca9783e43" candidate_uuid: "ef158262d7574ab998d2ead0bf1f300d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ff835b4f96b849cf934c9ea972c47293"
I20260812 06:54:52.581182 20787 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f177fe529afb44e6841e29fca9783e43" candidate_uuid: "ef158262d7574ab998d2ead0bf1f300d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1e464ffec54649d3af8fe62c2fe6076a"
I20260812 06:54:52.581301 20636 raft_consensus.cc:3037] T f177fe529afb44e6841e29fca9783e43 P ff835b4f96b849cf934c9ea972c47293 [term 0 FOLLOWER]: Advancing to term 1
I20260812 06:54:52.581422 20787 raft_consensus.cc:2379] T f177fe529afb44e6841e29fca9783e43 P 1e464ffec54649d3af8fe62c2fe6076a [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate ef158262d7574ab998d2ead0bf1f300d in current term 1: Already voted for candidate 1e464ffec54649d3af8fe62c2fe6076a in this term.
I20260812 06:54:52.581607 20635 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f177fe529afb44e6841e29fca9783e43" candidate_uuid: "1e464ffec54649d3af8fe62c2fe6076a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ff835b4f96b849cf934c9ea972c47293"
I20260812 06:54:52.582219 20476 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f177fe529afb44e6841e29fca9783e43" candidate_uuid: "1e464ffec54649d3af8fe62c2fe6076a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ef158262d7574ab998d2ead0bf1f300d"
I20260812 06:54:52.582391 20476 raft_consensus.cc:2379] T f177fe529afb44e6841e29fca9783e43 P ef158262d7574ab998d2ead0bf1f300d [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 1e464ffec54649d3af8fe62c2fe6076a in current term 1: Already voted for candidate ef158262d7574ab998d2ead0bf1f300d in this term.
I20260812 06:54:52.582711 20636 raft_consensus.cc:2454] T f177fe529afb44e6841e29fca9783e43 P ff835b4f96b849cf934c9ea972c47293 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ef158262d7574ab998d2ead0bf1f300d in term 1.
I20260812 06:54:52.582778 20715 leader_election.cc:302] T f177fe529afb44e6841e29fca9783e43 P 1e464ffec54649d3af8fe62c2fe6076a [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1e464ffec54649d3af8fe62c2fe6076a; no voters: ef158262d7574ab998d2ead0bf1f300d, ff835b4f96b849cf934c9ea972c47293
I20260812 06:54:52.582954 20404 leader_election.cc:302] T f177fe529afb44e6841e29fca9783e43 P ef158262d7574ab998d2ead0bf1f300d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ef158262d7574ab998d2ead0bf1f300d, ff835b4f96b849cf934c9ea972c47293; no voters: 1e464ffec54649d3af8fe62c2fe6076a
I20260812 06:54:52.583061 20864 raft_consensus.cc:2726] T f177fe529afb44e6841e29fca9783e43 P 1e464ffec54649d3af8fe62c2fe6076a [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260812 06:54:52.583122 20862 raft_consensus.cc:2781] T f177fe529afb44e6841e29fca9783e43 P ef158262d7574ab998d2ead0bf1f300d [term 1 FOLLOWER]: Leader election won for term 1
I20260812 06:54:52.583381 20862 raft_consensus.cc:686] T f177fe529afb44e6841e29fca9783e43 P ef158262d7574ab998d2ead0bf1f300d [term 1 LEADER]: Becoming Leader. State: Replica: ef158262d7574ab998d2ead0bf1f300d, State: Running, Role: LEADER
I20260812 06:54:52.583724 20862 consensus_queue.cc:237] T f177fe529afb44e6841e29fca9783e43 P ef158262d7574ab998d2ead0bf1f300d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef158262d7574ab998d2ead0bf1f300d" member_type: VOTER last_known_addr { host: "127.19.52.65" port: 37203 } } peers { permanent_uuid: "1e464ffec54649d3af8fe62c2fe6076a" member_type: VOTER last_known_addr { host: "127.19.52.67" port: 35041 } } peers { permanent_uuid: "ff835b4f96b849cf934c9ea972c47293" member_type: VOTER last_known_addr { host: "127.19.52.66" port: 37985 } }
I20260812 06:54:52.587316 20301 catalog_manager.cc:5495] T f177fe529afb44e6841e29fca9783e43 P ef158262d7574ab998d2ead0bf1f300d reported cstate change: term changed from 0 to 1, leader changed from <none> to ef158262d7574ab998d2ead0bf1f300d (127.19.52.65). New cstate: current_term: 1 leader_uuid: "ef158262d7574ab998d2ead0bf1f300d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef158262d7574ab998d2ead0bf1f300d" member_type: VOTER last_known_addr { host: "127.19.52.65" port: 37203 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1e464ffec54649d3af8fe62c2fe6076a" member_type: VOTER last_known_addr { host: "127.19.52.67" port: 35041 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ff835b4f96b849cf934c9ea972c47293" member_type: VOTER last_known_addr { host: "127.19.52.66" port: 37985 } health_report { overall_health: UNKNOWN } } }
Aug 12 06:54:52 dist-test-slave-65mx krb5kdc[20255](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517691, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.19.52.65@KRBTEST.COM
I20260812 06:54:52.621250 20852 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:54:52.606461 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42876 (local address 127.19.52.65:37203)
0812 06:54:52.606643 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0812 06:54:52.606648 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 06:54:52.606801 (+ 153us) server_negotiation.cc:407] Connection header received
0812 06:54:52.606977 (+ 176us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:54:52.606980 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:54:52.607040 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:54:52.607164 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:54:52.607862 (+ 698us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:54:52.608389 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:54:52.609317 (+ 928us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:54:52.609550 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:54:52.609657 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:54:52.617207 (+ 7550us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:54:52.617224 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:54:52.617240 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:54:52.617278 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:54:52.619164 (+ 1886us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:54:52.619713 (+ 549us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:54:52.619719 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:54:52.619720 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:54:52.619835 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:54:52.620172 (+ 337us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:54:52.620176 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:54:52.620177 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:54:52.620470 (+ 293us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:54:52.620731 (+ 261us) server_negotiation.cc:1035] Waiting for connection context
0812 06:54:52.620939 (+ 208us) server_negotiation.cc:299] Negotiation successful
0812 06:54:52.621088 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
Aug 12 06:54:52 dist-test-slave-65mx krb5kdc[20255](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517692, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 12 06:54:52 dist-test-slave-65mx krb5kdc[20255](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517692, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.19.52.65@KRBTEST.COM
I20260812 06:54:52.658160 20852 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:54:52.649130 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42888 (local address 127.19.52.65:37203)
0812 06:54:52.649339 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0812 06:54:52.649344 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 06:54:52.649414 (+ 70us) server_negotiation.cc:407] Connection header received
0812 06:54:52.649586 (+ 172us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:54:52.649589 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:54:52.649643 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:54:52.649725 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:54:52.650462 (+ 737us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:54:52.650987 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:54:52.651815 (+ 828us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:54:52.652105 (+ 290us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:54:52.652194 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:54:52.654790 (+ 2596us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:54:52.654793 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:54:52.654795 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:54:52.654823 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:54:52.656342 (+ 1519us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:54:52.656822 (+ 480us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:54:52.656824 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:54:52.656825 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:54:52.656913 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:54:52.657161 (+ 248us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:54:52.657163 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:54:52.657164 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:54:52.657324 (+ 160us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:54:52.657446 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0812 06:54:52.657754 (+ 308us) server_negotiation.cc:299] Negotiation successful
0812 06:54:52.657985 (+ 231us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":84}
Aug 12 06:54:52 dist-test-slave-65mx krb5kdc[20255](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517692, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
W20260812 06:54:52.679181 20530 tablet.cc:2369] T f177fe529afb44e6841e29fca9783e43 P ef158262d7574ab998d2ead0bf1f300d: Can't schedule compaction. Clean time has not been advanced past its initial value.
Aug 12 06:54:52 dist-test-slave-65mx krb5kdc[20255](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517692, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.19.52.65@KRBTEST.COM
I20260812 06:54:52.688306 20852 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:54:52.678862 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42896 (local address 127.19.52.65:37203)
0812 06:54:52.678999 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0812 06:54:52.679004 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 06:54:52.679343 (+ 339us) server_negotiation.cc:407] Connection header received
0812 06:54:52.679530 (+ 187us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:54:52.679533 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:54:52.679586 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:54:52.679668 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:54:52.680392 (+ 724us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:54:52.680976 (+ 584us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:54:52.681854 (+ 878us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:54:52.682066 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:54:52.682171 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:54:52.685011 (+ 2840us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:54:52.685015 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:54:52.685017 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:54:52.685073 (+ 56us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:54:52.686557 (+ 1484us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:54:52.687029 (+ 472us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:54:52.687033 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:54:52.687035 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:54:52.687123 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:54:52.687409 (+ 286us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:54:52.687412 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:54:52.687414 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:54:52.687602 (+ 188us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:54:52.687765 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0812 06:54:52.687997 (+ 232us) server_negotiation.cc:299] Negotiation successful
0812 06:54:52.688131 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
W20260812 06:54:52.690351 20431 tablet_service.cc:2908] Rejecting scan request for tablet f177fe529afb44e6841e29fca9783e43: Uninitialized: clean time has not yet been initialized
I20260812 06:54:52.692935 19665 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu with pid 20385
W20260812 06:54:52.696434 20840 tablet.cc:2369] T f177fe529afb44e6841e29fca9783e43 P 1e464ffec54649d3af8fe62c2fe6076a: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 06:54:52.697288 20691 tablet.cc:2369] T f177fe529afb44e6841e29fca9783e43 P ff835b4f96b849cf934c9ea972c47293: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 06:54:52.702668 19665 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu with pid 20536
I20260812 06:54:52.711427 19665 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu with pid 20695
I20260812 06:54:52.719826 19665 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu with pid 20269
2026-08-12T06:54:52Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnChecksum (2590 ms)
[ RUN ] SecurityITest.SmokeTestAsAuthorizedUser
Loading random data
Initializing database '/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786517687384214-19665-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 06:54:52 dist-test-slave-65mx krb5kdc[20884](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 06:54:52 dist-test-slave-65mx krb5kdc[20884](info): set up 2 sockets
Aug 12 06:54:52 dist-test-slave-65mx krb5kdc[20884](info): commencing operation
krb5kdc: starting...
W20260812 06:54:53.867383 19665 mini_kdc.cc:121] Time spent starting KDC: real 1.121s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786517687384214-19665-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786517687384214-19665-0/krb5kdc/test-user.keytab.
Aug 12 06:54:53 dist-test-slave-65mx krb5kdc[20884](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517693, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T06:54:53Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T06:54:53Z Disabled control of system clock
WARNING: no policy specified for kudu/127.19.52.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.52.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.52.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786517687384214-19665-0/krb5kdc/kudu_127.19.52.126.keytab.
Entry for principal kudu/127.19.52.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786517687384214-19665-0/krb5kdc/kudu_127.19.52.126.keytab.
I20260812 06:54:54.008909 19665 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu
/tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786517687384214-19665-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786517687384214-19665-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786517687384214-19665-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786517687384214-19665-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.52.126:34419
--webserver_interface=127.19.52.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786517687384214-19665-0/krb5kdc/kudu_127.19.52.126.keytab
--principal=kudu/127.19.52.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.52.84:44709
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.19.52.126:34419
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786517687384214-19665-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786517687384214-19665-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786517687384214-19665-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 06:54:54.125376 20899 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:54:54.125677 20899 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:54:54.125722 20899 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 06:54:54.129797 20899 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 06:54:54.129878 20899 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260812 06:54:54.129896 20899 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 06:54:54.129915 20899 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 06:54:54.129932 20899 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 06:54:54.129947 20899 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 06:54:54.135462 20899 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.52.84:44709
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786517687384214-19665-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786517687384214-19665-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.19.52.126:34419
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786517687384214-19665-0/krb5kdc/kudu_127.19.52.126.keytab
--principal=kudu/127.19.52.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.52.126:34419
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786517687384214-19665-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.19.52.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20899
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786517687384214-19665-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 06:15:54 UTC on ccycloud.quasar-rnfzce.root.comops.site
build id 4581989
I20260812 06:54:54.136641 20899 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:54:54.137705 20899 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 06:54:54.146673 20907 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:54:54.146735 20905 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:54:54.146905 20899 server_base.cc:1034] running on GCE node
W20260812 06:54:54.146657 20910 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:54:54.147621 20899 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:54:54.148890 20899 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:54:54.150092 20899 hybrid_clock.cc:648] HybridClock initialized: now 1786517694150065 us; error 43 us; skew 500 ppm
Aug 12 06:54:54 dist-test-slave-65mx krb5kdc[20884](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517694, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.52.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 06:54:54.153242 20899 init.cc:377] Logged in from keytab as kudu/127.19.52.126@KRBTEST.COM (short username kudu)
I20260812 06:54:54.154224 20899 webserver.cc:460] Webserver started at http://127.19.52.126:35019/ using document root <none> and password file <none>
I20260812 06:54:54.154834 20899 fs_manager.cc:359] Metadata directory not provided
I20260812 06:54:54.154932 20899 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:54:54.155187 20899 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 06:54:54.157487 20899 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786517687384214-19665-0/minicluster-data/master-0/data/instance:
uuid: "71ba5b0b680f4403a6860d61d029a0d9"
format_stamp: "Formatted at 2026-08-12 06:54:54 on dist-test-slave-65mx"
server_key: "011ddd166428762cef0a632eafcf56d5"
server_key_iv: "2c745241a55533ed9c6168f0a30d7453"
server_key_version: "encryptionkey@0"
I20260812 06:54:54.158097 20899 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786517687384214-19665-0/minicluster-data/master-0/wal/instance:
uuid: "71ba5b0b680f4403a6860d61d029a0d9"
format_stamp: "Formatted at 2026-08-12 06:54:54 on dist-test-slave-65mx"
server_key: "011ddd166428762cef0a632eafcf56d5"
server_key_iv: "2c745241a55533ed9c6168f0a30d7453"
server_key_version: "encryptionkey@0"
I20260812 06:54:54.162472 20899 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260812 06:54:54.165362 20917 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 06:54:54.166800 20899 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 06:54:54.166970 20899 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786517687384214-19665-0/minicluster-data/master-0/data,/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786517687384214-19665-0/minicluster-data/master-0/wal
uuid: "71ba5b0b680f4403a6860d61d029a0d9"
format_stamp: "Formatted at 2026-08-12 06:54:54 on dist-test-slave-65mx"
server_key: "011ddd166428762cef0a632eafcf56d5"
server_key_iv: "2c745241a55533ed9c6168f0a30d7453"
server_key_version: "encryptionkey@0"
I20260812 06:54:54.167101 20899 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786517687384214-19665-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786517687384214-19665-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786517687384214-19665-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:54:54.185336 20899 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 06:54:54.189410 20899 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 06:54:54.189653 20899 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:54:54.198786 20899 rpc_server.cc:307] RPC server started. Bound to: 127.19.52.126:34419
I20260812 06:54:54.198858 20986 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.52.126:34419 every 8 connection(s)
I20260812 06:54:54.199918 20899 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786517687384214-19665-0/minicluster-data/master-0/data/info.pb
I20260812 06:54:54.202924 20987 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 06:54:54.206947 19665 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu as pid 20899
I20260812 06:54:54.207121 19665 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786517687384214-19665-0/minicluster-data/master-0/wal/instance
I20260812 06:54:54.207401 19665 external_mini_cluster.cc:1428] Setting key 2b37f73c4e025c06c520490485e57cff
I20260812 06:54:54.209183 20987 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 71ba5b0b680f4403a6860d61d029a0d9: Bootstrap starting.
I20260812 06:54:54.211872 20987 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 71ba5b0b680f4403a6860d61d029a0d9: Neither blocks nor log segments found. Creating new log.
I20260812 06:54:54.212760 20987 log.cc:824] T 00000000000000000000000000000000 P 71ba5b0b680f4403a6860d61d029a0d9: Log is configured to *not* fsync() on all Append() calls
I20260812 06:54:54.215431 20987 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 71ba5b0b680f4403a6860d61d029a0d9: No bootstrap required, opened a new log
Aug 12 06:54:54 dist-test-slave-65mx krb5kdc[20884](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517693, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.19.52.126@KRBTEST.COM
I20260812 06:54:54.219859 20987 raft_consensus.cc:348] T 00000000000000000000000000000000 P 71ba5b0b680f4403a6860d61d029a0d9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "71ba5b0b680f4403a6860d61d029a0d9" member_type: VOTER last_known_addr { host: "127.19.52.126" port: 34419 } }
I20260812 06:54:54.220189 20987 raft_consensus.cc:374] T 00000000000000000000000000000000 P 71ba5b0b680f4403a6860d61d029a0d9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 06:54:54.220292 20987 raft_consensus.cc:729] T 00000000000000000000000000000000 P 71ba5b0b680f4403a6860d61d029a0d9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 71ba5b0b680f4403a6860d61d029a0d9, State: Initialized, Role: FOLLOWER
I20260812 06:54:54.220830 20987 consensus_queue.cc:260] T 00000000000000000000000000000000 P 71ba5b0b680f4403a6860d61d029a0d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "71ba5b0b680f4403a6860d61d029a0d9" member_type: VOTER last_known_addr { host: "127.19.52.126" port: 34419 } }
I20260812 06:54:54.221029 20987 raft_consensus.cc:388] T 00000000000000000000000000000000 P 71ba5b0b680f4403a6860d61d029a0d9 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 06:54:54.221110 20987 raft_consensus.cc:482] T 00000000000000000000000000000000 P 71ba5b0b680f4403a6860d61d029a0d9 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 06:54:54.221242 20987 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 71ba5b0b680f4403a6860d61d029a0d9 [term 0 FOLLOWER]: Advancing to term 1
I20260812 06:54:54.222427 20987 raft_consensus.cc:504] T 00000000000000000000000000000000 P 71ba5b0b680f4403a6860d61d029a0d9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "71ba5b0b680f4403a6860d61d029a0d9" member_type: VOTER last_known_addr { host: "127.19.52.126" port: 34419 } }
I20260812 06:54:54.222846 20987 leader_election.cc:302] T 00000000000000000000000000000000 P 71ba5b0b680f4403a6860d61d029a0d9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 71ba5b0b680f4403a6860d61d029a0d9; no voters:
I20260812 06:54:54.223284 20987 leader_election.cc:288] T 00000000000000000000000000000000 P 71ba5b0b680f4403a6860d61d029a0d9 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 06:54:54.223448 20992 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 71ba5b0b680f4403a6860d61d029a0d9 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 06:54:54.223714 20992 raft_consensus.cc:686] T 00000000000000000000000000000000 P 71ba5b0b680f4403a6860d61d029a0d9 [term 1 LEADER]: Becoming Leader. State: Replica: 71ba5b0b680f4403a6860d61d029a0d9, State: Running, Role: LEADER
I20260812 06:54:54.224097 20992 consensus_queue.cc:237] T 00000000000000000000000000000000 P 71ba5b0b680f4403a6860d61d029a0d9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "71ba5b0b680f4403a6860d61d029a0d9" member_type: VOTER last_known_addr { host: "127.19.52.126" port: 34419 } }
I20260812 06:54:54.224723 20990 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:54:54.208999 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33364 (local address 127.19.52.126:34419)
0812 06:54:54.209437 (+ 438us) server_negotiation.cc:206] Beginning negotiation
0812 06:54:54.209453 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0812 06:54:54.209490 (+ 37us) server_negotiation.cc:407] Connection header received
0812 06:54:54.210171 (+ 681us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:54:54.210189 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:54:54.210525 (+ 336us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:54:54.210818 (+ 293us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:54:54.211763 (+ 945us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:54:54.213026 (+ 1263us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:54:54.214160 (+ 1134us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:54:54.214720 (+ 560us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:54:54.214864 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:54:54.218092 (+ 3228us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:54:54.218098 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:54:54.218114 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:54:54.218144 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:54:54.220476 (+ 2332us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:54:54.221087 (+ 611us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:54:54.221094 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:54:54.221103 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:54:54.221315 (+ 212us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:54:54.221685 (+ 370us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:54:54.221688 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:54:54.221689 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:54:54.222141 (+ 452us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:54:54.222345 (+ 204us) server_negotiation.cc:1035] Waiting for connection context
0812 06:54:54.222888 (+ 543us) server_negotiation.cc:299] Negotiation successful
0812 06:54:54.223273 (+ 385us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":288,"thread_start_us":145,"threads_started":1}
I20260812 06:54:54.224762 20987 sys_catalog.cc:564] T 00000000000000000000000000000000 P 71ba5b0b680f4403a6860d61d029a0d9 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 06:54:54.225992 20993 sys_catalog.cc:455] T 00000000000000000000000000000000 P 71ba5b0b680f4403a6860d61d029a0d9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "71ba5b0b680f4403a6860d61d029a0d9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "71ba5b0b680f4403a6860d61d029a0d9" member_type: VOTER last_known_addr { host: "127.19.52.126" port: 34419 } } }
I20260812 06:54:54.226140 20993 sys_catalog.cc:458] T 00000000000000000000000000000000 P 71ba5b0b680f4403a6860d61d029a0d9 [sys.catalog]: This master's current role is: LEADER
I20260812 06:54:54.226470 20994 sys_catalog.cc:455] T 00000000000000000000000000000000 P 71ba5b0b680f4403a6860d61d029a0d9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 71ba5b0b680f4403a6860d61d029a0d9. Latest consensus state: current_term: 1 leader_uuid: "71ba5b0b680f4403a6860d61d029a0d9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "71ba5b0b680f4403a6860d61d029a0d9" member_type: VOTER last_known_addr { host: "127.19.52.126" port: 34419 } } }
I20260812 06:54:54.226578 20994 sys_catalog.cc:458] T 00000000000000000000000000000000 P 71ba5b0b680f4403a6860d61d029a0d9 [sys.catalog]: This master's current role is: LEADER
I20260812 06:54:54.227567 21005 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 06:54:54.230911 21005 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 06:54:54.237315 21005 catalog_manager.cc:1347] Generated new cluster ID: 01c5d9ed852b45d38b089366d5746f92
I20260812 06:54:54.237460 21005 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 06:54:54.246011 21005 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 06:54:54.247243 21005 catalog_manager.cc:1504] Loading token signing keys...
I20260812 06:54:54.256371 21005 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 71ba5b0b680f4403a6860d61d029a0d9: Generated new TSK 0
I20260812 06:54:54.257243 21005 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.19.52.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.52.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.52.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786517687384214-19665-0/krb5kdc/kudu_127.19.52.65.keytab.
Entry for principal kudu/127.19.52.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786517687384214-19665-0/krb5kdc/kudu_127.19.52.65.keytab.
I20260812 06:54:54.289179 19665 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu
/tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786517687384214-19665-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786517687384214-19665-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786517687384214-19665-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786517687384214-19665-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.52.65:0
--local_ip_for_outbound_sockets=127.19.52.65
--webserver_interface=127.19.52.65
--webserver_port=0
--tserver_master_addrs=127.19.52.126:34419
--keytab_file=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786517687384214-19665-0/krb5kdc/kudu_127.19.52.65.keytab
--principal=kudu/127.19.52.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.52.84:44709
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786517687384214-19665-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786517687384214-19665-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786517687384214-19665-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 06:54:54.416158 21020 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:54:54.416481 21020 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:54:54.416532 21020 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 06:54:54.421022 21020 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 06:54:54.421114 21020 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 06:54:54.421164 21020 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260812 06:54:54.421239 21020 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.52.65
I20260812 06:54:54.427721 21020 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.52.84:44709
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786517687384214-19665-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786517687384214-19665-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786517687384214-19665-0/krb5kdc/kudu_127.19.52.65.keytab
--principal=kudu/127.19.52.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.52.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786517687384214-19665-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.19.52.65
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.19.52.126:34419
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21020
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.52.65
--log_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786517687384214-19665-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 06:15:54 UTC on ccycloud.quasar-rnfzce.root.comops.site
build id 4581989
I20260812 06:54:54.429031 21020 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:54:54.430168 21020 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 06:54:54.440680 21029 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:54:54.440732 21031 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:54:54.440879 21020 server_base.cc:1034] running on GCE node
W20260812 06:54:54.440688 21028 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:54:54.442198 21020 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:54:54.442900 21020 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:54:54.444141 21020 hybrid_clock.cc:648] HybridClock initialized: now 1786517694444126 us; error 29 us; skew 500 ppm
Aug 12 06:54:54 dist-test-slave-65mx krb5kdc[20884](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517694, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.52.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 06:54:54.447594 21020 init.cc:377] Logged in from keytab as kudu/127.19.52.65@KRBTEST.COM (short username kudu)
I20260812 06:54:54.448640 21020 webserver.cc:460] Webserver started at http://127.19.52.65:39651/ using document root <none> and password file <none>
I20260812 06:54:54.449206 21020 fs_manager.cc:359] Metadata directory not provided
I20260812 06:54:54.449265 21020 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:54:54.449441 21020 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 06:54:54.451537 21020 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786517687384214-19665-0/minicluster-data/ts-0/data/instance:
uuid: "e80a7453f73c4bb2a6bcaac705d19736"
format_stamp: "Formatted at 2026-08-12 06:54:54 on dist-test-slave-65mx"
server_key: "a19e12d2c30d0e920d225873e696fc42"
server_key_iv: "a72e0ae1abe543d5831308924ae68d69"
server_key_version: "encryptionkey@0"
I20260812 06:54:54.452088 21020 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786517687384214-19665-0/minicluster-data/ts-0/wal/instance:
uuid: "e80a7453f73c4bb2a6bcaac705d19736"
format_stamp: "Formatted at 2026-08-12 06:54:54 on dist-test-slave-65mx"
server_key: "a19e12d2c30d0e920d225873e696fc42"
server_key_iv: "a72e0ae1abe543d5831308924ae68d69"
server_key_version: "encryptionkey@0"
I20260812 06:54:54.456095 21020 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260812 06:54:54.458896 21039 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 06:54:54.460321 21020 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260812 06:54:54.460436 21020 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786517687384214-19665-0/minicluster-data/ts-0/data,/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786517687384214-19665-0/minicluster-data/ts-0/wal
uuid: "e80a7453f73c4bb2a6bcaac705d19736"
format_stamp: "Formatted at 2026-08-12 06:54:54 on dist-test-slave-65mx"
server_key: "a19e12d2c30d0e920d225873e696fc42"
server_key_iv: "a72e0ae1abe543d5831308924ae68d69"
server_key_version: "encryptionkey@0"
I20260812 06:54:54.460611 21020 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786517687384214-19665-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786517687384214-19665-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786517687384214-19665-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:54:54.484681 21020 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 06:54:54.488231 21020 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 06:54:54.488405 21020 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:54:54.489959 21020 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 06:54:54.490021 21020 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 06:54:54.490089 21020 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 06:54:54.490129 21020 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 12 06:54:54 dist-test-slave-65mx krb5kdc[20884](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517694, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.52.65@KRBTEST.COM for kudu/127.19.52.126@KRBTEST.COM
I20260812 06:54:54.502285 21020 rpc_server.cc:307] RPC server started. Bound to: 127.19.52.65:46259
I20260812 06:54:54.502581 21170 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.52.65:46259 every 8 connection(s)
I20260812 06:54:54.503441 21020 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786517687384214-19665-0/minicluster-data/ts-0/data/info.pb
I20260812 06:54:54.505424 20990 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:54:54.490937 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.52.65:52851 (local address 127.19.52.126:34419)
0812 06:54:54.491104 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0812 06:54:54.491108 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 06:54:54.492205 (+ 1097us) server_negotiation.cc:407] Connection header received
0812 06:54:54.493395 (+ 1190us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:54:54.493401 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:54:54.493491 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:54:54.493599 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:54:54.495071 (+ 1472us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:54:54.495784 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:54:54.497376 (+ 1592us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:54:54.497737 (+ 361us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:54:54.497863 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:54:54.501239 (+ 3376us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:54:54.501243 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:54:54.501245 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:54:54.501284 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:54:54.503061 (+ 1777us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:54:54.503732 (+ 671us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:54:54.503735 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:54:54.503736 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:54:54.503823 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:54:54.504267 (+ 444us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:54:54.504272 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:54:54.504275 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:54:54.504510 (+ 235us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:54:54.504675 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0812 06:54:54.505181 (+ 506us) server_negotiation.cc:299] Negotiation successful
0812 06:54:54.505301 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260812 06:54:54.506278 21050 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:54:54.491527 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.126:34419 (local address 127.19.52.65:52851)
0812 06:54:54.492036 (+ 509us) negotiation.cc:107] Waiting for socket to connect
0812 06:54:54.492087 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0812 06:54:54.493099 (+ 1012us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:54:54.493961 (+ 862us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:54:54.493968 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:54:54.494317 (+ 349us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:54:54.494883 (+ 566us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:54:54.494898 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:54:54.495927 (+ 1029us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:54:54.495932 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:54:54.497217 (+ 1285us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:54:54.497235 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:54:54.497914 (+ 679us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:54:54.497918 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:54:54.498105 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:54:54.498844 (+ 739us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 06:54:54.498865 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:54:54.500971 (+ 2106us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 06:54:54.503196 (+ 2225us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:54:54.503202 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:54:54.503217 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:54:54.503592 (+ 375us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:54:54.503930 (+ 338us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:54:54.503933 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:54:54.503935 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:54:54.504120 (+ 185us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:54:54.504678 (+ 558us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 06:54:54.504684 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 06:54:54.504938 (+ 254us) client_negotiation.cc:769] Sending connection context
0812 06:54:54.505100 (+ 162us) client_negotiation.cc:240] Negotiation successful
0812 06:54:54.505376 (+ 276us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":310,"thread_start_us":143,"threads_started":1}
I20260812 06:54:54.507845 19665 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu as pid 21020
I20260812 06:54:54.507938 21171 heartbeater.cc:347] Connected to a master server at 127.19.52.126:34419
I20260812 06:54:54.507992 19665 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786517687384214-19665-0/minicluster-data/ts-0/wal/instance
I20260812 06:54:54.508090 21171 heartbeater.cc:464] Registering TS with master...
I20260812 06:54:54.508307 19665 external_mini_cluster.cc:1428] Setting key 8bb438f8e92724b827087259ccbcd668
I20260812 06:54:54.508751 21171 heartbeater.cc:511] Master 127.19.52.126:34419 requested a full tablet report, sending...
I20260812 06:54:54.510469 20939 ts_manager.cc:194] Registered new tserver with Master: e80a7453f73c4bb2a6bcaac705d19736 (127.19.52.65:46259)
I20260812 06:54:54.511888 20939 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.52.65@KRBTEST.COM'} at 127.19.52.65:52851
I20260812 06:54:54.519364 20990 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:54:54.510552 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.52.65:60953 (local address 127.19.52.126:34419)
0812 06:54:54.510683 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0812 06:54:54.510688 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 06:54:54.510930 (+ 242us) server_negotiation.cc:407] Connection header received
0812 06:54:54.511083 (+ 153us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:54:54.511087 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:54:54.511137 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:54:54.511224 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:54:54.511786 (+ 562us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:54:54.512512 (+ 726us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:54:54.513485 (+ 973us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:54:54.513746 (+ 261us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:54:54.513861 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:54:54.515543 (+ 1682us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:54:54.515546 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:54:54.515549 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:54:54.515573 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:54:54.517506 (+ 1933us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:54:54.518053 (+ 547us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:54:54.518056 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:54:54.518057 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:54:54.518142 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:54:54.518505 (+ 363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:54:54.518507 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:54:54.518508 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:54:54.518690 (+ 182us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:54:54.518774 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0812 06:54:54.519116 (+ 342us) server_negotiation.cc:299] Negotiation successful
0812 06:54:54.519240 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260812 06:54:54.519496 21178 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:54:54.510426 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.126:34419 (local address 127.19.52.65:60953)
0812 06:54:54.510840 (+ 414us) negotiation.cc:107] Waiting for socket to connect
0812 06:54:54.510852 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 06:54:54.510976 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:54:54.511241 (+ 265us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:54:54.511247 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:54:54.511502 (+ 255us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:54:54.511650 (+ 148us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:54:54.511660 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:54:54.512719 (+ 1059us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:54:54.512723 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:54:54.513353 (+ 630us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:54:54.513363 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:54:54.514013 (+ 650us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:54:54.514019 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:54:54.514140 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:54:54.514799 (+ 659us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 06:54:54.514816 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:54:54.515344 (+ 528us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 06:54:54.517633 (+ 2289us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:54:54.517640 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:54:54.517642 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:54:54.517939 (+ 297us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:54:54.518266 (+ 327us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:54:54.518272 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:54:54.518276 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:54:54.518389 (+ 113us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:54:54.518799 (+ 410us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 06:54:54.518805 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 06:54:54.518935 (+ 130us) client_negotiation.cc:769] Sending connection context
0812 06:54:54.519063 (+ 128us) client_negotiation.cc:240] Negotiation successful
0812 06:54:54.519252 (+ 189us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":343,"thread_start_us":114,"threads_started":1}
WARNING: no policy specified for kudu/127.19.52.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.52.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.52.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786517687384214-19665-0/krb5kdc/kudu_127.19.52.66.keytab.
Entry for principal kudu/127.19.52.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786517687384214-19665-0/krb5kdc/kudu_127.19.52.66.keytab.
I20260812 06:54:54.544711 19665 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu
/tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786517687384214-19665-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786517687384214-19665-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786517687384214-19665-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786517687384214-19665-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.52.66:0
--local_ip_for_outbound_sockets=127.19.52.66
--webserver_interface=127.19.52.66
--webserver_port=0
--tserver_master_addrs=127.19.52.126:34419
--keytab_file=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786517687384214-19665-0/krb5kdc/kudu_127.19.52.66.keytab
--principal=kudu/127.19.52.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.52.84:44709
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786517687384214-19665-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786517687384214-19665-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786517687384214-19665-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 06:54:54.663264 21181 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:54:54.663615 21181 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:54:54.663708 21181 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 06:54:54.668326 21181 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 06:54:54.668455 21181 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 06:54:54.668529 21181 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260812 06:54:54.668642 21181 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.52.66
I20260812 06:54:54.674515 21181 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.52.84:44709
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786517687384214-19665-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786517687384214-19665-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786517687384214-19665-0/krb5kdc/kudu_127.19.52.66.keytab
--principal=kudu/127.19.52.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.52.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786517687384214-19665-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.19.52.66
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.19.52.126:34419
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21181
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.52.66
--log_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786517687384214-19665-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 06:15:54 UTC on ccycloud.quasar-rnfzce.root.comops.site
build id 4581989
I20260812 06:54:54.675696 21181 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:54:54.676872 21181 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 06:54:54.687456 21186 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:54:54.687528 21187 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:54:54.687456 21191 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:54:54.687695 21181 server_base.cc:1034] running on GCE node
I20260812 06:54:54.688256 21181 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:54:54.689052 21181 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:54:54.690234 21181 hybrid_clock.cc:648] HybridClock initialized: now 1786517694690205 us; error 35 us; skew 500 ppm
Aug 12 06:54:54 dist-test-slave-65mx krb5kdc[20884](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517694, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.52.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 06:54:54.693296 21181 init.cc:377] Logged in from keytab as kudu/127.19.52.66@KRBTEST.COM (short username kudu)
I20260812 06:54:54.694303 21181 webserver.cc:460] Webserver started at http://127.19.52.66:38335/ using document root <none> and password file <none>
I20260812 06:54:54.694892 21181 fs_manager.cc:359] Metadata directory not provided
I20260812 06:54:54.694996 21181 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:54:54.695238 21181 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 06:54:54.697569 21181 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786517687384214-19665-0/minicluster-data/ts-1/data/instance:
uuid: "a7a637fb5800477db43eb0b6621d165a"
format_stamp: "Formatted at 2026-08-12 06:54:54 on dist-test-slave-65mx"
server_key: "837201c21ccce6f0a359d1ddba273e7a"
server_key_iv: "565f3be13534e261742598193cdff909"
server_key_version: "encryptionkey@0"
I20260812 06:54:54.698395 21181 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786517687384214-19665-0/minicluster-data/ts-1/wal/instance:
uuid: "a7a637fb5800477db43eb0b6621d165a"
format_stamp: "Formatted at 2026-08-12 06:54:54 on dist-test-slave-65mx"
server_key: "837201c21ccce6f0a359d1ddba273e7a"
server_key_iv: "565f3be13534e261742598193cdff909"
server_key_version: "encryptionkey@0"
I20260812 06:54:54.702940 21181 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.004s
I20260812 06:54:54.705766 21200 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 06:54:54.707221 21181 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 06:54:54.707429 21181 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786517687384214-19665-0/minicluster-data/ts-1/data,/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786517687384214-19665-0/minicluster-data/ts-1/wal
uuid: "a7a637fb5800477db43eb0b6621d165a"
format_stamp: "Formatted at 2026-08-12 06:54:54 on dist-test-slave-65mx"
server_key: "837201c21ccce6f0a359d1ddba273e7a"
server_key_iv: "565f3be13534e261742598193cdff909"
server_key_version: "encryptionkey@0"
I20260812 06:54:54.707574 21181 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786517687384214-19665-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786517687384214-19665-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786517687384214-19665-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:54:54.727692 21181 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 06:54:54.731374 21181 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 06:54:54.731627 21181 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:54:54.733343 21181 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 06:54:54.733440 21181 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 06:54:54.733520 21181 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 06:54:54.733575 21181 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 12 06:54:54 dist-test-slave-65mx krb5kdc[20884](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517694, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.52.66@KRBTEST.COM for kudu/127.19.52.126@KRBTEST.COM
I20260812 06:54:54.745450 21181 rpc_server.cc:307] RPC server started. Bound to: 127.19.52.66:43119
I20260812 06:54:54.745674 21331 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.52.66:43119 every 8 connection(s)
I20260812 06:54:54.746517 21181 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786517687384214-19665-0/minicluster-data/ts-1/data/info.pb
I20260812 06:54:54.749261 20990 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:54:54.734355 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.52.66:53585 (local address 127.19.52.126:34419)
0812 06:54:54.734558 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0812 06:54:54.734563 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 06:54:54.735502 (+ 939us) server_negotiation.cc:407] Connection header received
0812 06:54:54.736935 (+ 1433us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:54:54.736938 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:54:54.737011 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:54:54.737120 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:54:54.738534 (+ 1414us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:54:54.739088 (+ 554us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:54:54.740698 (+ 1610us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:54:54.740931 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:54:54.741054 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:54:54.744834 (+ 3780us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:54:54.744837 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:54:54.744839 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:54:54.744868 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:54:54.746649 (+ 1781us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:54:54.747383 (+ 734us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:54:54.747387 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:54:54.747389 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:54:54.747497 (+ 108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:54:54.747952 (+ 455us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:54:54.747956 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:54:54.747958 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:54:54.748175 (+ 217us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:54:54.748296 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0812 06:54:54.748939 (+ 643us) server_negotiation.cc:299] Negotiation successful
0812 06:54:54.749082 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260812 06:54:54.750140 21211 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:54:54.734774 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.126:34419 (local address 127.19.52.66:53585)
0812 06:54:54.735289 (+ 515us) negotiation.cc:107] Waiting for socket to connect
0812 06:54:54.735356 (+ 67us) client_negotiation.cc:174] Beginning negotiation
0812 06:54:54.736613 (+ 1257us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:54:54.737338 (+ 725us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:54:54.737349 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:54:54.737823 (+ 474us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:54:54.738362 (+ 539us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:54:54.738378 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:54:54.739242 (+ 864us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:54:54.739246 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:54:54.740519 (+ 1273us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:54:54.740530 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:54:54.741117 (+ 587us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:54:54.741121 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:54:54.741310 (+ 189us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:54:54.741983 (+ 673us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 06:54:54.742016 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:54:54.744618 (+ 2602us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 06:54:54.746797 (+ 2179us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:54:54.746803 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:54:54.746818 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:54:54.747237 (+ 419us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:54:54.747631 (+ 394us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:54:54.747634 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:54:54.747636 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:54:54.747834 (+ 198us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:54:54.748302 (+ 468us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 06:54:54.748311 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 06:54:54.748669 (+ 358us) client_negotiation.cc:769] Sending connection context
0812 06:54:54.748876 (+ 207us) client_negotiation.cc:240] Negotiation successful
0812 06:54:54.749230 (+ 354us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":273,"thread_start_us":108,"threads_started":1}
I20260812 06:54:54.751469 21332 heartbeater.cc:347] Connected to a master server at 127.19.52.126:34419
I20260812 06:54:54.751672 21332 heartbeater.cc:464] Registering TS with master...
I20260812 06:54:54.752205 21332 heartbeater.cc:511] Master 127.19.52.126:34419 requested a full tablet report, sending...
I20260812 06:54:54.753401 19665 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu as pid 21181
I20260812 06:54:54.753520 19665 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786517687384214-19665-0/minicluster-data/ts-1/wal/instance
I20260812 06:54:54.753522 20939 ts_manager.cc:194] Registered new tserver with Master: a7a637fb5800477db43eb0b6621d165a (127.19.52.66:43119)
I20260812 06:54:54.753822 19665 external_mini_cluster.cc:1428] Setting key a9582be836e6ccda8973fbf7900d1450
I20260812 06:54:54.754341 20939 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.52.66@KRBTEST.COM'} at 127.19.52.66:53585
WARNING: no policy specified for kudu/127.19.52.67@KRBTEST.COM; defaulting to no policy
I20260812 06:54:54.764065 20990 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:54:54.754704 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.52.66:50061 (local address 127.19.52.126:34419)
0812 06:54:54.754815 (+ 111us) server_negotiation.cc:206] Beginning negotiation
0812 06:54:54.754820 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 06:54:54.754834 (+ 14us) server_negotiation.cc:407] Connection header received
0812 06:54:54.754869 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:54:54.754871 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:54:54.754913 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:54:54.755014 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:54:54.757334 (+ 2320us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:54:54.758005 (+ 671us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:54:54.758857 (+ 852us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:54:54.759094 (+ 237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:54:54.759209 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:54:54.760577 (+ 1368us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:54:54.760581 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:54:54.760584 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:54:54.760611 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:54:54.762164 (+ 1553us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:54:54.762737 (+ 573us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:54:54.762742 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:54:54.762744 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:54:54.762836 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:54:54.763134 (+ 298us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:54:54.763138 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:54:54.763140 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:54:54.763376 (+ 236us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:54:54.763504 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0812 06:54:54.763791 (+ 287us) server_negotiation.cc:299] Negotiation successful
0812 06:54:54.763908 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260812 06:54:54.764161 21339 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:54:54.753987 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.126:34419 (local address 127.19.52.66:50061)
0812 06:54:54.754301 (+ 314us) negotiation.cc:107] Waiting for socket to connect
0812 06:54:54.754324 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0812 06:54:54.754422 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:54:54.755144 (+ 722us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:54:54.755147 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:54:54.757072 (+ 1925us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:54:54.757180 (+ 108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:54:54.757188 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:54:54.758156 (+ 968us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:54:54.758160 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:54:54.758729 (+ 569us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:54:54.758738 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:54:54.759221 (+ 483us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:54:54.759225 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:54:54.759325 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:54:54.759897 (+ 572us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 06:54:54.759909 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:54:54.760276 (+ 367us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 06:54:54.762298 (+ 2022us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:54:54.762303 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:54:54.762305 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:54:54.762612 (+ 307us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:54:54.762934 (+ 322us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:54:54.762945 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:54:54.762947 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:54:54.763037 (+ 90us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:54:54.763491 (+ 454us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 06:54:54.763520 (+ 29us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 06:54:54.763623 (+ 103us) client_negotiation.cc:769] Sending connection context
0812 06:54:54.763733 (+ 110us) client_negotiation.cc:240] Negotiation successful
0812 06:54:54.763898 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":198,"thread_start_us":114,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.52.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.52.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786517687384214-19665-0/krb5kdc/kudu_127.19.52.67.keytab.
Entry for principal kudu/127.19.52.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786517687384214-19665-0/krb5kdc/kudu_127.19.52.67.keytab.
I20260812 06:54:54.788637 19665 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu
/tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786517687384214-19665-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786517687384214-19665-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786517687384214-19665-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786517687384214-19665-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.52.67:0
--local_ip_for_outbound_sockets=127.19.52.67
--webserver_interface=127.19.52.67
--webserver_port=0
--tserver_master_addrs=127.19.52.126:34419
--keytab_file=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786517687384214-19665-0/krb5kdc/kudu_127.19.52.67.keytab
--principal=kudu/127.19.52.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.52.84:44709
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786517687384214-19665-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786517687384214-19665-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786517687384214-19665-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 06:54:54.909338 21343 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:54:54.909626 21343 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:54:54.909708 21343 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 06:54:54.913983 21343 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 06:54:54.914062 21343 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 06:54:54.914103 21343 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260812 06:54:54.914167 21343 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.52.67
I20260812 06:54:54.919884 21343 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.52.84:44709
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786517687384214-19665-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786517687384214-19665-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786517687384214-19665-0/krb5kdc/kudu_127.19.52.67.keytab
--principal=kudu/127.19.52.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.52.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786517687384214-19665-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.19.52.67
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.19.52.126:34419
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21343
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.52.67
--log_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786517687384214-19665-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 06:15:54 UTC on ccycloud.quasar-rnfzce.root.comops.site
build id 4581989
I20260812 06:54:54.921202 21343 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:54:54.922262 21343 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 06:54:54.932559 21349 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:54:54.932572 21350 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:54:54.932572 21353 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:54:54.932838 21343 server_base.cc:1034] running on GCE node
I20260812 06:54:54.933341 21343 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:54:54.934079 21343 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:54:54.935297 21343 hybrid_clock.cc:648] HybridClock initialized: now 1786517694935280 us; error 35 us; skew 500 ppm
Aug 12 06:54:54 dist-test-slave-65mx krb5kdc[20884](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517694, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.52.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 06:54:54.938905 21343 init.cc:377] Logged in from keytab as kudu/127.19.52.67@KRBTEST.COM (short username kudu)
I20260812 06:54:54.939963 21343 webserver.cc:460] Webserver started at http://127.19.52.67:37649/ using document root <none> and password file <none>
I20260812 06:54:54.940613 21343 fs_manager.cc:359] Metadata directory not provided
I20260812 06:54:54.940713 21343 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:54:54.940958 21343 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 06:54:54.943094 21343 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786517687384214-19665-0/minicluster-data/ts-2/data/instance:
uuid: "a1317386ad9141689a7b9f543db83e85"
format_stamp: "Formatted at 2026-08-12 06:54:54 on dist-test-slave-65mx"
server_key: "a1868f5a3c2d6f51be35d3843bb57242"
server_key_iv: "861a91ddb5b6081710ede1f374e6276b"
server_key_version: "encryptionkey@0"
I20260812 06:54:54.943701 21343 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786517687384214-19665-0/minicluster-data/ts-2/wal/instance:
uuid: "a1317386ad9141689a7b9f543db83e85"
format_stamp: "Formatted at 2026-08-12 06:54:54 on dist-test-slave-65mx"
server_key: "a1868f5a3c2d6f51be35d3843bb57242"
server_key_iv: "861a91ddb5b6081710ede1f374e6276b"
server_key_version: "encryptionkey@0"
I20260812 06:54:54.947685 21343 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.005s
I20260812 06:54:54.950405 21362 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 06:54:54.951761 21343 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260812 06:54:54.951928 21343 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786517687384214-19665-0/minicluster-data/ts-2/data,/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786517687384214-19665-0/minicluster-data/ts-2/wal
uuid: "a1317386ad9141689a7b9f543db83e85"
format_stamp: "Formatted at 2026-08-12 06:54:54 on dist-test-slave-65mx"
server_key: "a1868f5a3c2d6f51be35d3843bb57242"
server_key_iv: "861a91ddb5b6081710ede1f374e6276b"
server_key_version: "encryptionkey@0"
I20260812 06:54:54.952055 21343 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786517687384214-19665-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786517687384214-19665-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786517687384214-19665-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:54:54.981745 21343 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 06:54:54.985630 21343 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 06:54:54.985854 21343 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:54:54.987397 21343 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 06:54:54.987478 21343 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 06:54:54.987555 21343 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 06:54:54.987607 21343 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 12 06:54:54 dist-test-slave-65mx krb5kdc[20884](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517694, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.52.67@KRBTEST.COM for kudu/127.19.52.126@KRBTEST.COM
I20260812 06:54:54.999800 21343 rpc_server.cc:307] RPC server started. Bound to: 127.19.52.67:45375
I20260812 06:54:54.999894 21489 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.52.67:45375 every 8 connection(s)
I20260812 06:54:55.000821 21343 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786517687384214-19665-0/minicluster-data/ts-2/data/info.pb
I20260812 06:54:55.002769 20990 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:54:54.988663 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.52.67:34097 (local address 127.19.52.126:34419)
0812 06:54:54.988839 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0812 06:54:54.988844 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 06:54:54.989820 (+ 976us) server_negotiation.cc:407] Connection header received
0812 06:54:54.991199 (+ 1379us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:54:54.991207 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:54:54.991278 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:54:54.991345 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:54:54.993522 (+ 2177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:54:54.994152 (+ 630us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:54:54.995590 (+ 1438us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:54:54.995832 (+ 242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:54:54.995954 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:54:54.998750 (+ 2796us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:54:54.998758 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:54:54.998761 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:54:54.998787 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:54:55.000437 (+ 1650us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:54:55.001022 (+ 585us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:54:55.001025 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:54:55.001027 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:54:55.001115 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:54:55.001524 (+ 409us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:54:55.001528 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:54:55.001530 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:54:55.001750 (+ 220us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:54:55.001877 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0812 06:54:55.002487 (+ 610us) server_negotiation.cc:299] Negotiation successful
0812 06:54:55.002642 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260812 06:54:55.003574 21373 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:54:54.989056 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.126:34419 (local address 127.19.52.67:34097)
0812 06:54:54.989619 (+ 563us) negotiation.cc:107] Waiting for socket to connect
0812 06:54:54.989679 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0812 06:54:54.990919 (+ 1240us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:54:54.991804 (+ 885us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:54:54.991812 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:54:54.992410 (+ 598us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:54:54.993278 (+ 868us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:54:54.993300 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:54:54.994311 (+ 1011us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:54:54.994318 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:54:54.995452 (+ 1134us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:54:54.995462 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:54:54.996022 (+ 560us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:54:54.996027 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:54:54.996210 (+ 183us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:54:54.996976 (+ 766us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 06:54:54.996993 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:54:54.998568 (+ 1575us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 06:54:55.000601 (+ 2033us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:54:55.000607 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:54:55.000621 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:54:55.000925 (+ 304us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:54:55.001211 (+ 286us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:54:55.001215 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:54:55.001217 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:54:55.001389 (+ 172us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:54:55.001931 (+ 542us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 06:54:55.001937 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 06:54:55.002184 (+ 247us) client_negotiation.cc:769] Sending connection context
0812 06:54:55.002360 (+ 176us) client_negotiation.cc:240] Negotiation successful
0812 06:54:55.002663 (+ 303us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":297,"thread_start_us":122,"threads_started":1}
I20260812 06:54:55.004879 21491 heartbeater.cc:347] Connected to a master server at 127.19.52.126:34419
I20260812 06:54:55.005079 21491 heartbeater.cc:464] Registering TS with master...
I20260812 06:54:55.005617 21491 heartbeater.cc:511] Master 127.19.52.126:34419 requested a full tablet report, sending...
I20260812 06:54:55.007061 20939 ts_manager.cc:194] Registered new tserver with Master: a1317386ad9141689a7b9f543db83e85 (127.19.52.67:45375)
I20260812 06:54:55.007439 19665 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu as pid 21343
I20260812 06:54:55.007571 19665 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786517687384214-19665-0/minicluster-data/ts-2/wal/instance
I20260812 06:54:55.007871 19665 external_mini_cluster.cc:1428] Setting key 8baca5701607457b941ff9ae119f5868
I20260812 06:54:55.008461 20939 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.52.67@KRBTEST.COM'} at 127.19.52.67:34097
I20260812 06:54:55.010475 19665 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 06:54:55.017992 21497 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:54:55.007327 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.126:34419 (local address 127.19.52.67:52077)
0812 06:54:55.007706 (+ 379us) negotiation.cc:107] Waiting for socket to connect
0812 06:54:55.007720 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 06:54:55.007802 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:54:55.009323 (+ 1521us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:54:55.009326 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:54:55.009628 (+ 302us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:54:55.009788 (+ 160us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:54:55.009796 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:54:55.010834 (+ 1038us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:54:55.010841 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:54:55.011646 (+ 805us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:54:55.011657 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:54:55.012640 (+ 983us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:54:55.012643 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:54:55.012823 (+ 180us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:54:55.013532 (+ 709us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 06:54:55.013544 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:54:55.014209 (+ 665us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 06:54:55.016045 (+ 1836us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:54:55.016051 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:54:55.016057 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:54:55.016353 (+ 296us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:54:55.016789 (+ 436us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:54:55.016794 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:54:55.016798 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:54:55.016925 (+ 127us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:54:55.017489 (+ 564us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 06:54:55.017492 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 06:54:55.017590 (+ 98us) client_negotiation.cc:769] Sending connection context
0812 06:54:55.017692 (+ 102us) client_negotiation.cc:240] Negotiation successful
0812 06:54:55.017834 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":267,"thread_start_us":127,"threads_started":1}
I20260812 06:54:55.018211 20990 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:54:55.008960 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.52.67:52077 (local address 127.19.52.126:34419)
0812 06:54:55.009095 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0812 06:54:55.009120 (+ 25us) server_negotiation.cc:399] Waiting for connection header
0812 06:54:55.009136 (+ 16us) server_negotiation.cc:407] Connection header received
0812 06:54:55.009175 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:54:55.009178 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:54:55.009222 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:54:55.009308 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:54:55.009917 (+ 609us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:54:55.010659 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:54:55.011800 (+ 1141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:54:55.012201 (+ 401us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:54:55.012296 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:54:55.014397 (+ 2101us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:54:55.014400 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:54:55.014402 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:54:55.014426 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:54:55.015945 (+ 1519us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:54:55.016495 (+ 550us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:54:55.016501 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:54:55.016503 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:54:55.016681 (+ 178us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:54:55.017077 (+ 396us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:54:55.017082 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:54:55.017086 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:54:55.017344 (+ 258us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:54:55.017471 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0812 06:54:55.017788 (+ 317us) server_negotiation.cc:299] Negotiation successful
0812 06:54:55.017962 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
Aug 12 06:54:55 dist-test-slave-65mx krb5kdc[20884](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517695, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 12 06:54:55 dist-test-slave-65mx krb5kdc[20884](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517695, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.19.52.126@KRBTEST.COM
I20260812 06:54:55.042332 20990 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:54:55.032657 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33374 (local address 127.19.52.126:34419)
0812 06:54:55.032840 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0812 06:54:55.032844 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 06:54:55.032861 (+ 17us) server_negotiation.cc:407] Connection header received
0812 06:54:55.032907 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:54:55.032910 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:54:55.033004 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:54:55.033104 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:54:55.034007 (+ 903us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:54:55.034572 (+ 565us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:54:55.035428 (+ 856us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:54:55.035676 (+ 248us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:54:55.035772 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:54:55.038841 (+ 3069us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:54:55.038844 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:54:55.038846 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:54:55.038877 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:54:55.040508 (+ 1631us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:54:55.041037 (+ 529us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:54:55.041040 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:54:55.041041 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:54:55.041142 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:54:55.041425 (+ 283us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:54:55.041428 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:54:55.041430 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:54:55.041640 (+ 210us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:54:55.041759 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0812 06:54:55.042069 (+ 310us) server_negotiation.cc:299] Negotiation successful
0812 06:54:55.042193 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260812 06:54:55.046283 20939 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:33374:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 06:54:55.050122 20939 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 06:54:55.065487 21510 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:54:55.059614 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40262 (local address 127.19.52.66:43119)
0812 06:54:55.060786 (+ 1172us) server_negotiation.cc:206] Beginning negotiation
0812 06:54:55.060798 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0812 06:54:55.060824 (+ 26us) server_negotiation.cc:407] Connection header received
0812 06:54:55.060907 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:54:55.060913 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:54:55.061129 (+ 216us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:54:55.061291 (+ 162us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 06:54:55.061637 (+ 346us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:54:55.062446 (+ 809us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:54:55.064022 (+ 1576us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:54:55.064771 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:54:55.064884 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:54:55.065059 (+ 175us) server_negotiation.cc:1035] Waiting for connection context
0812 06:54:55.065235 (+ 176us) server_negotiation.cc:299] Negotiation successful
0812 06:54:55.065334 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1047,"thread_start_us":112,"threads_started":1}
I20260812 06:54:55.065477 21509 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:54:55.059436 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.66:43119 (local address 127.0.0.1:40262)
0812 06:54:55.059989 (+ 553us) negotiation.cc:107] Waiting for socket to connect
0812 06:54:55.060057 (+ 68us) client_negotiation.cc:174] Beginning negotiation
0812 06:54:55.060191 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:54:55.061308 (+ 1117us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:54:55.061314 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:54:55.061338 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 06:54:55.061487 (+ 149us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:54:55.061494 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:54:55.062567 (+ 1073us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:54:55.062572 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:54:55.063832 (+ 1260us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:54:55.063842 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:54:55.064899 (+ 1057us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:54:55.064903 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:54:55.065025 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:54:55.065067 (+ 42us) client_negotiation.cc:769] Sending connection context
0812 06:54:55.065185 (+ 118us) client_negotiation.cc:240] Negotiation successful
0812 06:54:55.065309 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":194,"mutex_wait_us":3,"thread_start_us":100,"threads_started":1}
I20260812 06:54:55.066298 21511 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:54:55.059835 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.67:45375 (local address 127.0.0.1:53288)
0812 06:54:55.060470 (+ 635us) negotiation.cc:107] Waiting for socket to connect
0812 06:54:55.060485 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 06:54:55.060628 (+ 143us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:54:55.061486 (+ 858us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:54:55.061489 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:54:55.061504 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 06:54:55.061607 (+ 103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:54:55.061619 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:54:55.063122 (+ 1503us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:54:55.063125 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:54:55.064496 (+ 1371us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:54:55.064519 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:54:55.065840 (+ 1321us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:54:55.065846 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:54:55.065965 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:54:55.065990 (+ 25us) client_negotiation.cc:769] Sending connection context
0812 06:54:55.066060 (+ 70us) client_negotiation.cc:240] Negotiation successful
0812 06:54:55.066138 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":539,"mutex_wait_us":129,"thread_start_us":109,"threads_started":1}
I20260812 06:54:55.066632 21514 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:54:55.060640 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54764 (local address 127.19.52.65:46259)
0812 06:54:55.061015 (+ 375us) server_negotiation.cc:206] Beginning negotiation
0812 06:54:55.061021 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 06:54:55.061039 (+ 18us) server_negotiation.cc:407] Connection header received
0812 06:54:55.061105 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:54:55.061110 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:54:55.061283 (+ 173us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:54:55.061403 (+ 120us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 06:54:55.061827 (+ 424us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:54:55.062979 (+ 1152us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:54:55.064651 (+ 1672us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:54:55.065755 (+ 1104us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:54:55.065873 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:54:55.066010 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0812 06:54:55.066369 (+ 359us) server_negotiation.cc:299] Negotiation successful
0812 06:54:55.066495 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":290,"thread_start_us":90,"threads_started":1}
I20260812 06:54:55.066666 21512 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:54:55.060194 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.65:46259 (local address 127.0.0.1:54764)
0812 06:54:55.060807 (+ 613us) negotiation.cc:107] Waiting for socket to connect
0812 06:54:55.060819 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 06:54:55.060896 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:54:55.061536 (+ 640us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:54:55.061538 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:54:55.061585 (+ 47us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 06:54:55.061687 (+ 102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:54:55.061695 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:54:55.063122 (+ 1427us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:54:55.063125 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:54:55.064520 (+ 1395us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:54:55.064526 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:54:55.065864 (+ 1338us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:54:55.065867 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:54:55.066137 (+ 270us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:54:55.066159 (+ 22us) client_negotiation.cc:769] Sending connection context
0812 06:54:55.066224 (+ 65us) client_negotiation.cc:240] Negotiation successful
0812 06:54:55.066281 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":556,"mutex_wait_us":146,"thread_start_us":94,"threads_started":1}
I20260812 06:54:55.067005 21513 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:54:55.060231 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53288 (local address 127.19.52.67:45375)
0812 06:54:55.060856 (+ 625us) server_negotiation.cc:206] Beginning negotiation
0812 06:54:55.060871 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0812 06:54:55.060894 (+ 23us) server_negotiation.cc:407] Connection header received
0812 06:54:55.060974 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:54:55.060979 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:54:55.061183 (+ 204us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:54:55.061353 (+ 170us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 06:54:55.061734 (+ 381us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:54:55.062979 (+ 1245us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:54:55.064707 (+ 1728us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:54:55.065478 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:54:55.065600 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:54:55.065760 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0812 06:54:55.066803 (+ 1043us) server_negotiation.cc:299] Negotiation successful
0812 06:54:55.066899 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":516,"thread_start_us":93,"threads_started":1}
I20260812 06:54:55.069144 21259 tablet_service.cc:1467] Processing CreateTablet for tablet d6fb2ed78eac44e5b4310202be325426 (DEFAULT_TABLE table=test-table [id=0dbf9df1829e4e4a82758e96e15d3535]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 06:54:55.069437 21418 tablet_service.cc:1467] Processing CreateTablet for tablet d6fb2ed78eac44e5b4310202be325426 (DEFAULT_TABLE table=test-table [id=0dbf9df1829e4e4a82758e96e15d3535]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 06:54:55.070272 21259 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d6fb2ed78eac44e5b4310202be325426. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 06:54:55.070616 21418 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d6fb2ed78eac44e5b4310202be325426. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 06:54:55.072263 21096 tablet_service.cc:1467] Processing CreateTablet for tablet d6fb2ed78eac44e5b4310202be325426 (DEFAULT_TABLE table=test-table [id=0dbf9df1829e4e4a82758e96e15d3535]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 06:54:55.073190 21096 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d6fb2ed78eac44e5b4310202be325426. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 06:54:55.078264 21515 tablet_bootstrap.cc:492] T d6fb2ed78eac44e5b4310202be325426 P a1317386ad9141689a7b9f543db83e85: Bootstrap starting.
I20260812 06:54:55.079630 21517 tablet_bootstrap.cc:492] T d6fb2ed78eac44e5b4310202be325426 P e80a7453f73c4bb2a6bcaac705d19736: Bootstrap starting.
I20260812 06:54:55.081029 21515 tablet_bootstrap.cc:654] T d6fb2ed78eac44e5b4310202be325426 P a1317386ad9141689a7b9f543db83e85: Neither blocks nor log segments found. Creating new log.
I20260812 06:54:55.081764 21517 tablet_bootstrap.cc:654] T d6fb2ed78eac44e5b4310202be325426 P e80a7453f73c4bb2a6bcaac705d19736: Neither blocks nor log segments found. Creating new log.
I20260812 06:54:55.082418 21515 log.cc:824] T d6fb2ed78eac44e5b4310202be325426 P a1317386ad9141689a7b9f543db83e85: Log is configured to *not* fsync() on all Append() calls
I20260812 06:54:55.082508 21517 log.cc:824] T d6fb2ed78eac44e5b4310202be325426 P e80a7453f73c4bb2a6bcaac705d19736: Log is configured to *not* fsync() on all Append() calls
I20260812 06:54:55.084246 21518 tablet_bootstrap.cc:492] T d6fb2ed78eac44e5b4310202be325426 P a7a637fb5800477db43eb0b6621d165a: Bootstrap starting.
I20260812 06:54:55.084635 21517 tablet_bootstrap.cc:492] T d6fb2ed78eac44e5b4310202be325426 P e80a7453f73c4bb2a6bcaac705d19736: No bootstrap required, opened a new log
I20260812 06:54:55.084643 21515 tablet_bootstrap.cc:492] T d6fb2ed78eac44e5b4310202be325426 P a1317386ad9141689a7b9f543db83e85: No bootstrap required, opened a new log
I20260812 06:54:55.084841 21515 ts_tablet_manager.cc:1397] T d6fb2ed78eac44e5b4310202be325426 P a1317386ad9141689a7b9f543db83e85: Time spent bootstrapping tablet: real 0.007s user 0.004s sys 0.001s
I20260812 06:54:55.084841 21517 ts_tablet_manager.cc:1397] T d6fb2ed78eac44e5b4310202be325426 P e80a7453f73c4bb2a6bcaac705d19736: Time spent bootstrapping tablet: real 0.005s user 0.005s sys 0.000s
I20260812 06:54:55.086436 21518 tablet_bootstrap.cc:654] T d6fb2ed78eac44e5b4310202be325426 P a7a637fb5800477db43eb0b6621d165a: Neither blocks nor log segments found. Creating new log.
I20260812 06:54:55.087196 21518 log.cc:824] T d6fb2ed78eac44e5b4310202be325426 P a7a637fb5800477db43eb0b6621d165a: Log is configured to *not* fsync() on all Append() calls
I20260812 06:54:55.089397 21518 tablet_bootstrap.cc:492] T d6fb2ed78eac44e5b4310202be325426 P a7a637fb5800477db43eb0b6621d165a: No bootstrap required, opened a new log
I20260812 06:54:55.089622 21518 ts_tablet_manager.cc:1397] T d6fb2ed78eac44e5b4310202be325426 P a7a637fb5800477db43eb0b6621d165a: Time spent bootstrapping tablet: real 0.007s user 0.004s sys 0.000s
I20260812 06:54:55.089723 21517 raft_consensus.cc:348] T d6fb2ed78eac44e5b4310202be325426 P e80a7453f73c4bb2a6bcaac705d19736 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a7a637fb5800477db43eb0b6621d165a" member_type: VOTER last_known_addr { host: "127.19.52.66" port: 43119 } } peers { permanent_uuid: "e80a7453f73c4bb2a6bcaac705d19736" member_type: VOTER last_known_addr { host: "127.19.52.65" port: 46259 } } peers { permanent_uuid: "a1317386ad9141689a7b9f543db83e85" member_type: VOTER last_known_addr { host: "127.19.52.67" port: 45375 } }
I20260812 06:54:55.089722 21515 raft_consensus.cc:348] T d6fb2ed78eac44e5b4310202be325426 P a1317386ad9141689a7b9f543db83e85 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a7a637fb5800477db43eb0b6621d165a" member_type: VOTER last_known_addr { host: "127.19.52.66" port: 43119 } } peers { permanent_uuid: "e80a7453f73c4bb2a6bcaac705d19736" member_type: VOTER last_known_addr { host: "127.19.52.65" port: 46259 } } peers { permanent_uuid: "a1317386ad9141689a7b9f543db83e85" member_type: VOTER last_known_addr { host: "127.19.52.67" port: 45375 } }
I20260812 06:54:55.090032 21517 raft_consensus.cc:374] T d6fb2ed78eac44e5b4310202be325426 P e80a7453f73c4bb2a6bcaac705d19736 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 06:54:55.090042 21515 raft_consensus.cc:374] T d6fb2ed78eac44e5b4310202be325426 P a1317386ad9141689a7b9f543db83e85 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 06:54:55.090101 21517 raft_consensus.cc:729] T d6fb2ed78eac44e5b4310202be325426 P e80a7453f73c4bb2a6bcaac705d19736 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e80a7453f73c4bb2a6bcaac705d19736, State: Initialized, Role: FOLLOWER
I20260812 06:54:55.090111 21515 raft_consensus.cc:729] T d6fb2ed78eac44e5b4310202be325426 P a1317386ad9141689a7b9f543db83e85 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a1317386ad9141689a7b9f543db83e85, State: Initialized, Role: FOLLOWER
I20260812 06:54:55.090600 21517 consensus_queue.cc:260] T d6fb2ed78eac44e5b4310202be325426 P e80a7453f73c4bb2a6bcaac705d19736 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a7a637fb5800477db43eb0b6621d165a" member_type: VOTER last_known_addr { host: "127.19.52.66" port: 43119 } } peers { permanent_uuid: "e80a7453f73c4bb2a6bcaac705d19736" member_type: VOTER last_known_addr { host: "127.19.52.65" port: 46259 } } peers { permanent_uuid: "a1317386ad9141689a7b9f543db83e85" member_type: VOTER last_known_addr { host: "127.19.52.67" port: 45375 } }
I20260812 06:54:55.090598 21515 consensus_queue.cc:260] T d6fb2ed78eac44e5b4310202be325426 P a1317386ad9141689a7b9f543db83e85 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a7a637fb5800477db43eb0b6621d165a" member_type: VOTER last_known_addr { host: "127.19.52.66" port: 43119 } } peers { permanent_uuid: "e80a7453f73c4bb2a6bcaac705d19736" member_type: VOTER last_known_addr { host: "127.19.52.65" port: 46259 } } peers { permanent_uuid: "a1317386ad9141689a7b9f543db83e85" member_type: VOTER last_known_addr { host: "127.19.52.67" port: 45375 } }
I20260812 06:54:55.091358 21171 heartbeater.cc:503] Master 127.19.52.126:34419 was elected leader, sending a full tablet report...
I20260812 06:54:55.091668 21491 heartbeater.cc:503] Master 127.19.52.126:34419 was elected leader, sending a full tablet report...
I20260812 06:54:55.091681 21515 ts_tablet_manager.cc:1428] T d6fb2ed78eac44e5b4310202be325426 P a1317386ad9141689a7b9f543db83e85: Time spent starting tablet: real 0.007s user 0.002s sys 0.004s
I20260812 06:54:55.092417 21517 ts_tablet_manager.cc:1428] T d6fb2ed78eac44e5b4310202be325426 P e80a7453f73c4bb2a6bcaac705d19736: Time spent starting tablet: real 0.008s user 0.007s sys 0.000s
I20260812 06:54:55.093659 21518 raft_consensus.cc:348] T d6fb2ed78eac44e5b4310202be325426 P a7a637fb5800477db43eb0b6621d165a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a7a637fb5800477db43eb0b6621d165a" member_type: VOTER last_known_addr { host: "127.19.52.66" port: 43119 } } peers { permanent_uuid: "e80a7453f73c4bb2a6bcaac705d19736" member_type: VOTER last_known_addr { host: "127.19.52.65" port: 46259 } } peers { permanent_uuid: "a1317386ad9141689a7b9f543db83e85" member_type: VOTER last_known_addr { host: "127.19.52.67" port: 45375 } }
I20260812 06:54:55.093971 21518 raft_consensus.cc:374] T d6fb2ed78eac44e5b4310202be325426 P a7a637fb5800477db43eb0b6621d165a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 06:54:55.094031 21518 raft_consensus.cc:729] T d6fb2ed78eac44e5b4310202be325426 P a7a637fb5800477db43eb0b6621d165a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a7a637fb5800477db43eb0b6621d165a, State: Initialized, Role: FOLLOWER
I20260812 06:54:55.094511 21518 consensus_queue.cc:260] T d6fb2ed78eac44e5b4310202be325426 P a7a637fb5800477db43eb0b6621d165a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a7a637fb5800477db43eb0b6621d165a" member_type: VOTER last_known_addr { host: "127.19.52.66" port: 43119 } } peers { permanent_uuid: "e80a7453f73c4bb2a6bcaac705d19736" member_type: VOTER last_known_addr { host: "127.19.52.65" port: 46259 } } peers { permanent_uuid: "a1317386ad9141689a7b9f543db83e85" member_type: VOTER last_known_addr { host: "127.19.52.67" port: 45375 } }
I20260812 06:54:55.095472 21332 heartbeater.cc:503] Master 127.19.52.126:34419 was elected leader, sending a full tablet report...
I20260812 06:54:55.095605 21518 ts_tablet_manager.cc:1428] T d6fb2ed78eac44e5b4310202be325426 P a7a637fb5800477db43eb0b6621d165a: Time spent starting tablet: real 0.006s user 0.003s sys 0.004s
W20260812 06:54:55.247880 21333 tablet.cc:2369] T d6fb2ed78eac44e5b4310202be325426 P a7a637fb5800477db43eb0b6621d165a: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 06:54:55.251825 21492 tablet.cc:2369] T d6fb2ed78eac44e5b4310202be325426 P a1317386ad9141689a7b9f543db83e85: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 06:54:55.255443 21172 tablet.cc:2369] T d6fb2ed78eac44e5b4310202be325426 P e80a7453f73c4bb2a6bcaac705d19736: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 06:54:55.300700 21523 raft_consensus.cc:482] T d6fb2ed78eac44e5b4310202be325426 P a1317386ad9141689a7b9f543db83e85 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 06:54:55.300956 21523 raft_consensus.cc:504] T d6fb2ed78eac44e5b4310202be325426 P a1317386ad9141689a7b9f543db83e85 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a7a637fb5800477db43eb0b6621d165a" member_type: VOTER last_known_addr { host: "127.19.52.66" port: 43119 } } peers { permanent_uuid: "e80a7453f73c4bb2a6bcaac705d19736" member_type: VOTER last_known_addr { host: "127.19.52.65" port: 46259 } } peers { permanent_uuid: "a1317386ad9141689a7b9f543db83e85" member_type: VOTER last_known_addr { host: "127.19.52.67" port: 45375 } }
I20260812 06:54:55.302258 21523 leader_election.cc:288] T d6fb2ed78eac44e5b4310202be325426 P a1317386ad9141689a7b9f543db83e85 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a7a637fb5800477db43eb0b6621d165a (127.19.52.66:43119), e80a7453f73c4bb2a6bcaac705d19736 (127.19.52.65:46259)
I20260812 06:54:55.306310 21510 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:54:55.302646 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.52.67:57647 (local address 127.19.52.66:43119)
0812 06:54:55.302780 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0812 06:54:55.302783 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 06:54:55.302819 (+ 36us) server_negotiation.cc:407] Connection header received
0812 06:54:55.302911 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:54:55.302918 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:54:55.302981 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:54:55.303089 (+ 108us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 06:54:55.303392 (+ 303us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:54:55.303912 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:54:55.305205 (+ 1293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:54:55.305718 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:54:55.305871 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:54:55.305943 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0812 06:54:55.306111 (+ 168us) server_negotiation.cc:299] Negotiation successful
0812 06:54:55.306185 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260812 06:54:55.306310 21373 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:54:55.302541 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.66:43119 (local address 127.19.52.67:57647)
0812 06:54:55.302684 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0812 06:54:55.302702 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0812 06:54:55.302809 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:54:55.303105 (+ 296us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:54:55.303109 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:54:55.303130 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 06:54:55.303268 (+ 138us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:54:55.303275 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:54:55.304080 (+ 805us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:54:55.304086 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:54:55.305077 (+ 991us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:54:55.305087 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:54:55.305844 (+ 757us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:54:55.305848 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:54:55.305970 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:54:55.306000 (+ 30us) client_negotiation.cc:769] Sending connection context
0812 06:54:55.306093 (+ 93us) client_negotiation.cc:240] Negotiation successful
0812 06:54:55.306171 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":45}
I20260812 06:54:55.307031 21525 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:54:55.302854 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.65:46259 (local address 127.19.52.67:51883)
0812 06:54:55.303333 (+ 479us) negotiation.cc:107] Waiting for socket to connect
0812 06:54:55.303344 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 06:54:55.303461 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:54:55.303693 (+ 232us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:54:55.303696 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:54:55.303712 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 06:54:55.304205 (+ 493us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:54:55.304211 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:54:55.305015 (+ 804us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:54:55.305021 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:54:55.305866 (+ 845us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:54:55.305878 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:54:55.306661 (+ 783us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:54:55.306665 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:54:55.306767 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:54:55.306794 (+ 27us) client_negotiation.cc:769] Sending connection context
0812 06:54:55.306873 (+ 79us) client_negotiation.cc:240] Negotiation successful
0812 06:54:55.306932 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":427,"thread_start_us":182,"threads_started":1}
I20260812 06:54:55.307232 21514 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:54:55.303042 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.52.67:51883 (local address 127.19.52.65:46259)
0812 06:54:55.303206 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0812 06:54:55.303211 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 06:54:55.303425 (+ 214us) server_negotiation.cc:407] Connection header received
0812 06:54:55.303549 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:54:55.303556 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:54:55.303599 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:54:55.303683 (+ 84us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 06:54:55.304346 (+ 663us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:54:55.304903 (+ 557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:54:55.305995 (+ 1092us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:54:55.306532 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:54:55.306652 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:54:55.306697 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0812 06:54:55.306998 (+ 301us) server_negotiation.cc:299] Negotiation successful
0812 06:54:55.307092 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260812 06:54:55.307196 21281 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d6fb2ed78eac44e5b4310202be325426" candidate_uuid: "a1317386ad9141689a7b9f543db83e85" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a7a637fb5800477db43eb0b6621d165a" is_pre_election: true
I20260812 06:54:55.307600 21281 raft_consensus.cc:2454] T d6fb2ed78eac44e5b4310202be325426 P a7a637fb5800477db43eb0b6621d165a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a1317386ad9141689a7b9f543db83e85 in term 0.
I20260812 06:54:55.307978 21118 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d6fb2ed78eac44e5b4310202be325426" candidate_uuid: "a1317386ad9141689a7b9f543db83e85" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e80a7453f73c4bb2a6bcaac705d19736" is_pre_election: true
I20260812 06:54:55.308218 21366 leader_election.cc:302] T d6fb2ed78eac44e5b4310202be325426 P a1317386ad9141689a7b9f543db83e85 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a1317386ad9141689a7b9f543db83e85, a7a637fb5800477db43eb0b6621d165a; no voters:
I20260812 06:54:55.308297 21118 raft_consensus.cc:2454] T d6fb2ed78eac44e5b4310202be325426 P e80a7453f73c4bb2a6bcaac705d19736 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a1317386ad9141689a7b9f543db83e85 in term 0.
I20260812 06:54:55.308571 21523 raft_consensus.cc:2781] T d6fb2ed78eac44e5b4310202be325426 P a1317386ad9141689a7b9f543db83e85 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 06:54:55.308708 21523 raft_consensus.cc:482] T d6fb2ed78eac44e5b4310202be325426 P a1317386ad9141689a7b9f543db83e85 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 06:54:55.308815 21523 raft_consensus.cc:3037] T d6fb2ed78eac44e5b4310202be325426 P a1317386ad9141689a7b9f543db83e85 [term 0 FOLLOWER]: Advancing to term 1
I20260812 06:54:55.310050 21523 raft_consensus.cc:504] T d6fb2ed78eac44e5b4310202be325426 P a1317386ad9141689a7b9f543db83e85 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a7a637fb5800477db43eb0b6621d165a" member_type: VOTER last_known_addr { host: "127.19.52.66" port: 43119 } } peers { permanent_uuid: "e80a7453f73c4bb2a6bcaac705d19736" member_type: VOTER last_known_addr { host: "127.19.52.65" port: 46259 } } peers { permanent_uuid: "a1317386ad9141689a7b9f543db83e85" member_type: VOTER last_known_addr { host: "127.19.52.67" port: 45375 } }
I20260812 06:54:55.310457 21523 leader_election.cc:288] T d6fb2ed78eac44e5b4310202be325426 P a1317386ad9141689a7b9f543db83e85 [CANDIDATE]: Term 1 election: Requested vote from peers a7a637fb5800477db43eb0b6621d165a (127.19.52.66:43119), e80a7453f73c4bb2a6bcaac705d19736 (127.19.52.65:46259)
I20260812 06:54:55.310916 21281 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d6fb2ed78eac44e5b4310202be325426" candidate_uuid: "a1317386ad9141689a7b9f543db83e85" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a7a637fb5800477db43eb0b6621d165a"
I20260812 06:54:55.310959 21118 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d6fb2ed78eac44e5b4310202be325426" candidate_uuid: "a1317386ad9141689a7b9f543db83e85" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e80a7453f73c4bb2a6bcaac705d19736"
I20260812 06:54:55.311066 21281 raft_consensus.cc:3037] T d6fb2ed78eac44e5b4310202be325426 P a7a637fb5800477db43eb0b6621d165a [term 0 FOLLOWER]: Advancing to term 1
I20260812 06:54:55.311100 21118 raft_consensus.cc:3037] T d6fb2ed78eac44e5b4310202be325426 P e80a7453f73c4bb2a6bcaac705d19736 [term 0 FOLLOWER]: Advancing to term 1
I20260812 06:54:55.312320 21118 raft_consensus.cc:2454] T d6fb2ed78eac44e5b4310202be325426 P e80a7453f73c4bb2a6bcaac705d19736 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a1317386ad9141689a7b9f543db83e85 in term 1.
I20260812 06:54:55.312362 21281 raft_consensus.cc:2454] T d6fb2ed78eac44e5b4310202be325426 P a7a637fb5800477db43eb0b6621d165a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a1317386ad9141689a7b9f543db83e85 in term 1.
I20260812 06:54:55.312680 21366 leader_election.cc:302] T d6fb2ed78eac44e5b4310202be325426 P a1317386ad9141689a7b9f543db83e85 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a1317386ad9141689a7b9f543db83e85, e80a7453f73c4bb2a6bcaac705d19736; no voters:
I20260812 06:54:55.312896 21523 raft_consensus.cc:2781] T d6fb2ed78eac44e5b4310202be325426 P a1317386ad9141689a7b9f543db83e85 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 06:54:55.313175 21523 raft_consensus.cc:686] T d6fb2ed78eac44e5b4310202be325426 P a1317386ad9141689a7b9f543db83e85 [term 1 LEADER]: Becoming Leader. State: Replica: a1317386ad9141689a7b9f543db83e85, State: Running, Role: LEADER
I20260812 06:54:55.313529 21523 consensus_queue.cc:237] T d6fb2ed78eac44e5b4310202be325426 P a1317386ad9141689a7b9f543db83e85 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a7a637fb5800477db43eb0b6621d165a" member_type: VOTER last_known_addr { host: "127.19.52.66" port: 43119 } } peers { permanent_uuid: "e80a7453f73c4bb2a6bcaac705d19736" member_type: VOTER last_known_addr { host: "127.19.52.65" port: 46259 } } peers { permanent_uuid: "a1317386ad9141689a7b9f543db83e85" member_type: VOTER last_known_addr { host: "127.19.52.67" port: 45375 } }
I20260812 06:54:55.316917 20938 catalog_manager.cc:5495] T d6fb2ed78eac44e5b4310202be325426 P a1317386ad9141689a7b9f543db83e85 reported cstate change: term changed from 0 to 1, leader changed from <none> to a1317386ad9141689a7b9f543db83e85 (127.19.52.67). New cstate: current_term: 1 leader_uuid: "a1317386ad9141689a7b9f543db83e85" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a7a637fb5800477db43eb0b6621d165a" member_type: VOTER last_known_addr { host: "127.19.52.66" port: 43119 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e80a7453f73c4bb2a6bcaac705d19736" member_type: VOTER last_known_addr { host: "127.19.52.65" port: 46259 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a1317386ad9141689a7b9f543db83e85" member_type: VOTER last_known_addr { host: "127.19.52.67" port: 45375 } health_report { overall_health: HEALTHY } } }
I20260812 06:54:55.367892 21513 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:54:55.363838 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37998 (local address 127.19.52.67:45375)
0812 06:54:55.363981 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0812 06:54:55.363986 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 06:54:55.364090 (+ 104us) server_negotiation.cc:407] Connection header received
0812 06:54:55.364348 (+ 258us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:54:55.364350 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:54:55.364408 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:54:55.364530 (+ 122us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 06:54:55.364884 (+ 354us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:54:55.365396 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:54:55.366363 (+ 967us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:54:55.366569 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:54:55.366652 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:54:55.367016 (+ 364us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 06:54:55.367481 (+ 465us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 06:54:55.367572 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0812 06:54:55.367714 (+ 142us) server_negotiation.cc:299] Negotiation successful
0812 06:54:55.367771 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260812 06:54:55.374720 21118 raft_consensus.cc:1261] T d6fb2ed78eac44e5b4310202be325426 P e80a7453f73c4bb2a6bcaac705d19736 [term 1 FOLLOWER]: Refusing update from remote peer a1317386ad9141689a7b9f543db83e85: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 06:54:55.375626 21523 consensus_queue.cc:1035] T d6fb2ed78eac44e5b4310202be325426 P a1317386ad9141689a7b9f543db83e85 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e80a7453f73c4bb2a6bcaac705d19736" member_type: VOTER last_known_addr { host: "127.19.52.65" port: 46259 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 06:54:55.376755 21281 raft_consensus.cc:1261] T d6fb2ed78eac44e5b4310202be325426 P a7a637fb5800477db43eb0b6621d165a [term 1 FOLLOWER]: Refusing update from remote peer a1317386ad9141689a7b9f543db83e85: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 06:54:55.377346 21523 consensus_queue.cc:1035] T d6fb2ed78eac44e5b4310202be325426 P a1317386ad9141689a7b9f543db83e85 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a7a637fb5800477db43eb0b6621d165a" member_type: VOTER last_known_addr { host: "127.19.52.66" port: 43119 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 06:54:55.383926 21530 mvcc.cc:205] Tried to move back new op lower bound from 7317576480246259712 to 7317576480006647808. Current Snapshot: MvccSnapshot[applied={T|T < 7317576480246259712}]
I20260812 06:54:55.398855 20939 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:33374:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 06:54:55.399122 20939 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:33374:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 06:54:55.402230 20939 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 71ba5b0b680f4403a6860d61d029a0d9: Sending DeleteTablet for 3 replicas of tablet d6fb2ed78eac44e5b4310202be325426
I20260812 06:54:55.403268 21096 tablet_service.cc:1514] Processing DeleteTablet for tablet d6fb2ed78eac44e5b4310202be325426 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 06:54:55 UTC) from {username='kudu', principal='kudu/127.19.52.126@KRBTEST.COM'} at 127.0.0.1:54764
I20260812 06:54:55.403338 21418 tablet_service.cc:1514] Processing DeleteTablet for tablet d6fb2ed78eac44e5b4310202be325426 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 06:54:55 UTC) from {username='kudu', principal='kudu/127.19.52.126@KRBTEST.COM'} at 127.0.0.1:53288
I20260812 06:54:55.403983 21538 tablet_replica.cc:330] T d6fb2ed78eac44e5b4310202be325426 P e80a7453f73c4bb2a6bcaac705d19736: stopping tablet replica
I20260812 06:54:55.404153 21259 tablet_service.cc:1514] Processing DeleteTablet for tablet d6fb2ed78eac44e5b4310202be325426 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 06:54:55 UTC) from {username='kudu', principal='kudu/127.19.52.126@KRBTEST.COM'} at 127.0.0.1:40262
I20260812 06:54:55.404351 21538 raft_consensus.cc:2229] T d6fb2ed78eac44e5b4310202be325426 P e80a7453f73c4bb2a6bcaac705d19736 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 06:54:55.404490 20939 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:33374:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260812 06:54:55.404735 21538 raft_consensus.cc:2258] T d6fb2ed78eac44e5b4310202be325426 P e80a7453f73c4bb2a6bcaac705d19736 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 06:54:55.404740 21539 tablet_replica.cc:330] T d6fb2ed78eac44e5b4310202be325426 P a1317386ad9141689a7b9f543db83e85: stopping tablet replica
W20260812 06:54:55.404973 20939 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 06:54:55.405136 21539 raft_consensus.cc:2229] T d6fb2ed78eac44e5b4310202be325426 P a1317386ad9141689a7b9f543db83e85 [term 1 LEADER]: Raft consensus shutting down.
I20260812 06:54:55.405622 21539 raft_consensus.cc:2258] T d6fb2ed78eac44e5b4310202be325426 P a1317386ad9141689a7b9f543db83e85 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 06:54:55.406337 21541 tablet_replica.cc:330] T d6fb2ed78eac44e5b4310202be325426 P a7a637fb5800477db43eb0b6621d165a: stopping tablet replica
I20260812 06:54:55.406445 21538 ts_tablet_manager.cc:1905] T d6fb2ed78eac44e5b4310202be325426 P e80a7453f73c4bb2a6bcaac705d19736: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 06:54:55.406800 21541 raft_consensus.cc:2229] T d6fb2ed78eac44e5b4310202be325426 P a7a637fb5800477db43eb0b6621d165a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 06:54:55.407166 21541 raft_consensus.cc:2258] T d6fb2ed78eac44e5b4310202be325426 P a7a637fb5800477db43eb0b6621d165a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 06:54:55.409112 21539 ts_tablet_manager.cc:1905] T d6fb2ed78eac44e5b4310202be325426 P a1317386ad9141689a7b9f543db83e85: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 06:54:55.410149 21538 ts_tablet_manager.cc:1918] T d6fb2ed78eac44e5b4310202be325426 P e80a7453f73c4bb2a6bcaac705d19736: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 06:54:55.410271 21538 log.cc:1196] T d6fb2ed78eac44e5b4310202be325426 P e80a7453f73c4bb2a6bcaac705d19736: Deleting WAL directory at /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786517687384214-19665-0/minicluster-data/ts-0/wal/wals/d6fb2ed78eac44e5b4310202be325426
I20260812 06:54:55.410660 21538 ts_tablet_manager.cc:1939] T d6fb2ed78eac44e5b4310202be325426 P e80a7453f73c4bb2a6bcaac705d19736: Deleting consensus metadata
I20260812 06:54:55.412278 21539 ts_tablet_manager.cc:1918] T d6fb2ed78eac44e5b4310202be325426 P a1317386ad9141689a7b9f543db83e85: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 06:54:55.412467 21539 log.cc:1196] T d6fb2ed78eac44e5b4310202be325426 P a1317386ad9141689a7b9f543db83e85: Deleting WAL directory at /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786517687384214-19665-0/minicluster-data/ts-2/wal/wals/d6fb2ed78eac44e5b4310202be325426
I20260812 06:54:55.412384 20921 catalog_manager.cc:4841] TS e80a7453f73c4bb2a6bcaac705d19736 (127.19.52.65:46259): tablet d6fb2ed78eac44e5b4310202be325426 (table test-table [id=0dbf9df1829e4e4a82758e96e15d3535]) successfully deleted
I20260812 06:54:55.412959 21539 ts_tablet_manager.cc:1939] T d6fb2ed78eac44e5b4310202be325426 P a1317386ad9141689a7b9f543db83e85: Deleting consensus metadata
I20260812 06:54:55.414228 21541 ts_tablet_manager.cc:1905] T d6fb2ed78eac44e5b4310202be325426 P a7a637fb5800477db43eb0b6621d165a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 06:54:55.414601 21418 tablet_service.cc:1467] Processing CreateTablet for tablet 1ab31d852d934ff2966f5f1af100b466 (DEFAULT_TABLE table=test-table [id=71e174c7534e42cdab5f5d67174d34c6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 06:54:55.414896 21418 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1ab31d852d934ff2966f5f1af100b466. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 06:54:55.417028 21541 ts_tablet_manager.cc:1918] T d6fb2ed78eac44e5b4310202be325426 P a7a637fb5800477db43eb0b6621d165a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 06:54:55.417153 21541 log.cc:1196] T d6fb2ed78eac44e5b4310202be325426 P a7a637fb5800477db43eb0b6621d165a: Deleting WAL directory at /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786517687384214-19665-0/minicluster-data/ts-1/wal/wals/d6fb2ed78eac44e5b4310202be325426
I20260812 06:54:55.417441 21541 ts_tablet_manager.cc:1939] T d6fb2ed78eac44e5b4310202be325426 P a7a637fb5800477db43eb0b6621d165a: Deleting consensus metadata
I20260812 06:54:55.418578 21259 tablet_service.cc:1467] Processing CreateTablet for tablet 1ab31d852d934ff2966f5f1af100b466 (DEFAULT_TABLE table=test-table [id=71e174c7534e42cdab5f5d67174d34c6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 06:54:55.418833 21259 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1ab31d852d934ff2966f5f1af100b466. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 06:54:55.419649 20919 catalog_manager.cc:4841] TS a1317386ad9141689a7b9f543db83e85 (127.19.52.67:45375): tablet d6fb2ed78eac44e5b4310202be325426 (table test-table [id=0dbf9df1829e4e4a82758e96e15d3535]) successfully deleted
I20260812 06:54:55.420127 20921 catalog_manager.cc:4841] TS a7a637fb5800477db43eb0b6621d165a (127.19.52.66:43119): tablet d6fb2ed78eac44e5b4310202be325426 (table test-table [id=0dbf9df1829e4e4a82758e96e15d3535]) successfully deleted
I20260812 06:54:55.420279 21096 tablet_service.cc:1467] Processing CreateTablet for tablet 1ab31d852d934ff2966f5f1af100b466 (DEFAULT_TABLE table=test-table [id=71e174c7534e42cdab5f5d67174d34c6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 06:54:55.420609 21096 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1ab31d852d934ff2966f5f1af100b466. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 06:54:55.420845 21515 tablet_bootstrap.cc:492] T 1ab31d852d934ff2966f5f1af100b466 P a1317386ad9141689a7b9f543db83e85: Bootstrap starting.
I20260812 06:54:55.421396 21518 tablet_bootstrap.cc:492] T 1ab31d852d934ff2966f5f1af100b466 P a7a637fb5800477db43eb0b6621d165a: Bootstrap starting.
I20260812 06:54:55.421914 21515 tablet_bootstrap.cc:654] T 1ab31d852d934ff2966f5f1af100b466 P a1317386ad9141689a7b9f543db83e85: Neither blocks nor log segments found. Creating new log.
I20260812 06:54:55.422629 21518 tablet_bootstrap.cc:654] T 1ab31d852d934ff2966f5f1af100b466 P a7a637fb5800477db43eb0b6621d165a: Neither blocks nor log segments found. Creating new log.
I20260812 06:54:55.423208 21517 tablet_bootstrap.cc:492] T 1ab31d852d934ff2966f5f1af100b466 P e80a7453f73c4bb2a6bcaac705d19736: Bootstrap starting.
I20260812 06:54:55.424983 21517 tablet_bootstrap.cc:654] T 1ab31d852d934ff2966f5f1af100b466 P e80a7453f73c4bb2a6bcaac705d19736: Neither blocks nor log segments found. Creating new log.
I20260812 06:54:55.425024 21515 tablet_bootstrap.cc:492] T 1ab31d852d934ff2966f5f1af100b466 P a1317386ad9141689a7b9f543db83e85: No bootstrap required, opened a new log
I20260812 06:54:55.425110 21515 ts_tablet_manager.cc:1397] T 1ab31d852d934ff2966f5f1af100b466 P a1317386ad9141689a7b9f543db83e85: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.001s
I20260812 06:54:55.425228 21518 tablet_bootstrap.cc:492] T 1ab31d852d934ff2966f5f1af100b466 P a7a637fb5800477db43eb0b6621d165a: No bootstrap required, opened a new log
I20260812 06:54:55.425357 21518 ts_tablet_manager.cc:1397] T 1ab31d852d934ff2966f5f1af100b466 P a7a637fb5800477db43eb0b6621d165a: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.001s
I20260812 06:54:55.425729 21515 raft_consensus.cc:348] T 1ab31d852d934ff2966f5f1af100b466 P a1317386ad9141689a7b9f543db83e85 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e80a7453f73c4bb2a6bcaac705d19736" member_type: VOTER last_known_addr { host: "127.19.52.65" port: 46259 } } peers { permanent_uuid: "a7a637fb5800477db43eb0b6621d165a" member_type: VOTER last_known_addr { host: "127.19.52.66" port: 43119 } } peers { permanent_uuid: "a1317386ad9141689a7b9f543db83e85" member_type: VOTER last_known_addr { host: "127.19.52.67" port: 45375 } }
I20260812 06:54:55.425858 21515 raft_consensus.cc:374] T 1ab31d852d934ff2966f5f1af100b466 P a1317386ad9141689a7b9f543db83e85 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 06:54:55.425890 21515 raft_consensus.cc:729] T 1ab31d852d934ff2966f5f1af100b466 P a1317386ad9141689a7b9f543db83e85 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a1317386ad9141689a7b9f543db83e85, State: Initialized, Role: FOLLOWER
I20260812 06:54:55.426043 21515 consensus_queue.cc:260] T 1ab31d852d934ff2966f5f1af100b466 P a1317386ad9141689a7b9f543db83e85 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e80a7453f73c4bb2a6bcaac705d19736" member_type: VOTER last_known_addr { host: "127.19.52.65" port: 46259 } } peers { permanent_uuid: "a7a637fb5800477db43eb0b6621d165a" member_type: VOTER last_known_addr { host: "127.19.52.66" port: 43119 } } peers { permanent_uuid: "a1317386ad9141689a7b9f543db83e85" member_type: VOTER last_known_addr { host: "127.19.52.67" port: 45375 } }
I20260812 06:54:55.426090 21518 raft_consensus.cc:348] T 1ab31d852d934ff2966f5f1af100b466 P a7a637fb5800477db43eb0b6621d165a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e80a7453f73c4bb2a6bcaac705d19736" member_type: VOTER last_known_addr { host: "127.19.52.65" port: 46259 } } peers { permanent_uuid: "a7a637fb5800477db43eb0b6621d165a" member_type: VOTER last_known_addr { host: "127.19.52.66" port: 43119 } } peers { permanent_uuid: "a1317386ad9141689a7b9f543db83e85" member_type: VOTER last_known_addr { host: "127.19.52.67" port: 45375 } }
I20260812 06:54:55.426208 21518 raft_consensus.cc:374] T 1ab31d852d934ff2966f5f1af100b466 P a7a637fb5800477db43eb0b6621d165a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 06:54:55.426244 21518 raft_consensus.cc:729] T 1ab31d852d934ff2966f5f1af100b466 P a7a637fb5800477db43eb0b6621d165a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a7a637fb5800477db43eb0b6621d165a, State: Initialized, Role: FOLLOWER
I20260812 06:54:55.426302 21515 ts_tablet_manager.cc:1428] T 1ab31d852d934ff2966f5f1af100b466 P a1317386ad9141689a7b9f543db83e85: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260812 06:54:55.426375 21518 consensus_queue.cc:260] T 1ab31d852d934ff2966f5f1af100b466 P a7a637fb5800477db43eb0b6621d165a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e80a7453f73c4bb2a6bcaac705d19736" member_type: VOTER last_known_addr { host: "127.19.52.65" port: 46259 } } peers { permanent_uuid: "a7a637fb5800477db43eb0b6621d165a" member_type: VOTER last_known_addr { host: "127.19.52.66" port: 43119 } } peers { permanent_uuid: "a1317386ad9141689a7b9f543db83e85" member_type: VOTER last_known_addr { host: "127.19.52.67" port: 45375 } }
I20260812 06:54:55.426637 21518 ts_tablet_manager.cc:1428] T 1ab31d852d934ff2966f5f1af100b466 P a7a637fb5800477db43eb0b6621d165a: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260812 06:54:55.428326 21517 tablet_bootstrap.cc:492] T 1ab31d852d934ff2966f5f1af100b466 P e80a7453f73c4bb2a6bcaac705d19736: No bootstrap required, opened a new log
I20260812 06:54:55.428426 21517 ts_tablet_manager.cc:1397] T 1ab31d852d934ff2966f5f1af100b466 P e80a7453f73c4bb2a6bcaac705d19736: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.003s
I20260812 06:54:55.429052 21517 raft_consensus.cc:348] T 1ab31d852d934ff2966f5f1af100b466 P e80a7453f73c4bb2a6bcaac705d19736 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e80a7453f73c4bb2a6bcaac705d19736" member_type: VOTER last_known_addr { host: "127.19.52.65" port: 46259 } } peers { permanent_uuid: "a7a637fb5800477db43eb0b6621d165a" member_type: VOTER last_known_addr { host: "127.19.52.66" port: 43119 } } peers { permanent_uuid: "a1317386ad9141689a7b9f543db83e85" member_type: VOTER last_known_addr { host: "127.19.52.67" port: 45375 } }
I20260812 06:54:55.429188 21517 raft_consensus.cc:374] T 1ab31d852d934ff2966f5f1af100b466 P e80a7453f73c4bb2a6bcaac705d19736 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 06:54:55.429229 21517 raft_consensus.cc:729] T 1ab31d852d934ff2966f5f1af100b466 P e80a7453f73c4bb2a6bcaac705d19736 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e80a7453f73c4bb2a6bcaac705d19736, State: Initialized, Role: FOLLOWER
I20260812 06:54:55.429353 21517 consensus_queue.cc:260] T 1ab31d852d934ff2966f5f1af100b466 P e80a7453f73c4bb2a6bcaac705d19736 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e80a7453f73c4bb2a6bcaac705d19736" member_type: VOTER last_known_addr { host: "127.19.52.65" port: 46259 } } peers { permanent_uuid: "a7a637fb5800477db43eb0b6621d165a" member_type: VOTER last_known_addr { host: "127.19.52.66" port: 43119 } } peers { permanent_uuid: "a1317386ad9141689a7b9f543db83e85" member_type: VOTER last_known_addr { host: "127.19.52.67" port: 45375 } }
I20260812 06:54:55.429690 21517 ts_tablet_manager.cc:1428] T 1ab31d852d934ff2966f5f1af100b466 P e80a7453f73c4bb2a6bcaac705d19736: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260812 06:54:55.637403 21526 raft_consensus.cc:482] T 1ab31d852d934ff2966f5f1af100b466 P a1317386ad9141689a7b9f543db83e85 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 06:54:55.637568 21526 raft_consensus.cc:504] T 1ab31d852d934ff2966f5f1af100b466 P a1317386ad9141689a7b9f543db83e85 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e80a7453f73c4bb2a6bcaac705d19736" member_type: VOTER last_known_addr { host: "127.19.52.65" port: 46259 } } peers { permanent_uuid: "a7a637fb5800477db43eb0b6621d165a" member_type: VOTER last_known_addr { host: "127.19.52.66" port: 43119 } } peers { permanent_uuid: "a1317386ad9141689a7b9f543db83e85" member_type: VOTER last_known_addr { host: "127.19.52.67" port: 45375 } }
I20260812 06:54:55.637990 21526 leader_election.cc:288] T 1ab31d852d934ff2966f5f1af100b466 P a1317386ad9141689a7b9f543db83e85 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e80a7453f73c4bb2a6bcaac705d19736 (127.19.52.65:46259), a7a637fb5800477db43eb0b6621d165a (127.19.52.66:43119)
I20260812 06:54:55.638659 21118 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1ab31d852d934ff2966f5f1af100b466" candidate_uuid: "a1317386ad9141689a7b9f543db83e85" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e80a7453f73c4bb2a6bcaac705d19736" is_pre_election: true
I20260812 06:54:55.638742 21281 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1ab31d852d934ff2966f5f1af100b466" candidate_uuid: "a1317386ad9141689a7b9f543db83e85" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a7a637fb5800477db43eb0b6621d165a" is_pre_election: true
I20260812 06:54:55.638864 21118 raft_consensus.cc:2454] T 1ab31d852d934ff2966f5f1af100b466 P e80a7453f73c4bb2a6bcaac705d19736 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a1317386ad9141689a7b9f543db83e85 in term 0.
I20260812 06:54:55.638875 21281 raft_consensus.cc:2454] T 1ab31d852d934ff2966f5f1af100b466 P a7a637fb5800477db43eb0b6621d165a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a1317386ad9141689a7b9f543db83e85 in term 0.
I20260812 06:54:55.639248 21366 leader_election.cc:302] T 1ab31d852d934ff2966f5f1af100b466 P a1317386ad9141689a7b9f543db83e85 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a1317386ad9141689a7b9f543db83e85, a7a637fb5800477db43eb0b6621d165a; no voters:
I20260812 06:54:55.639496 21526 raft_consensus.cc:2781] T 1ab31d852d934ff2966f5f1af100b466 P a1317386ad9141689a7b9f543db83e85 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 06:54:55.639576 21526 raft_consensus.cc:482] T 1ab31d852d934ff2966f5f1af100b466 P a1317386ad9141689a7b9f543db83e85 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 06:54:55.639663 21526 raft_consensus.cc:3037] T 1ab31d852d934ff2966f5f1af100b466 P a1317386ad9141689a7b9f543db83e85 [term 0 FOLLOWER]: Advancing to term 1
I20260812 06:54:55.640915 21526 raft_consensus.cc:504] T 1ab31d852d934ff2966f5f1af100b466 P a1317386ad9141689a7b9f543db83e85 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e80a7453f73c4bb2a6bcaac705d19736" member_type: VOTER last_known_addr { host: "127.19.52.65" port: 46259 } } peers { permanent_uuid: "a7a637fb5800477db43eb0b6621d165a" member_type: VOTER last_known_addr { host: "127.19.52.66" port: 43119 } } peers { permanent_uuid: "a1317386ad9141689a7b9f543db83e85" member_type: VOTER last_known_addr { host: "127.19.52.67" port: 45375 } }
I20260812 06:54:55.641330 21526 leader_election.cc:288] T 1ab31d852d934ff2966f5f1af100b466 P a1317386ad9141689a7b9f543db83e85 [CANDIDATE]: Term 1 election: Requested vote from peers e80a7453f73c4bb2a6bcaac705d19736 (127.19.52.65:46259), a7a637fb5800477db43eb0b6621d165a (127.19.52.66:43119)
I20260812 06:54:55.641882 21281 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1ab31d852d934ff2966f5f1af100b466" candidate_uuid: "a1317386ad9141689a7b9f543db83e85" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a7a637fb5800477db43eb0b6621d165a"
I20260812 06:54:55.641881 21118 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1ab31d852d934ff2966f5f1af100b466" candidate_uuid: "a1317386ad9141689a7b9f543db83e85" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e80a7453f73c4bb2a6bcaac705d19736"
I20260812 06:54:55.642057 21281 raft_consensus.cc:3037] T 1ab31d852d934ff2966f5f1af100b466 P a7a637fb5800477db43eb0b6621d165a [term 0 FOLLOWER]: Advancing to term 1
I20260812 06:54:55.642086 21118 raft_consensus.cc:3037] T 1ab31d852d934ff2966f5f1af100b466 P e80a7453f73c4bb2a6bcaac705d19736 [term 0 FOLLOWER]: Advancing to term 1
I20260812 06:54:55.643467 21118 raft_consensus.cc:2454] T 1ab31d852d934ff2966f5f1af100b466 P e80a7453f73c4bb2a6bcaac705d19736 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a1317386ad9141689a7b9f543db83e85 in term 1.
I20260812 06:54:55.643633 21281 raft_consensus.cc:2454] T 1ab31d852d934ff2966f5f1af100b466 P a7a637fb5800477db43eb0b6621d165a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a1317386ad9141689a7b9f543db83e85 in term 1.
I20260812 06:54:55.643913 21366 leader_election.cc:302] T 1ab31d852d934ff2966f5f1af100b466 P a1317386ad9141689a7b9f543db83e85 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a1317386ad9141689a7b9f543db83e85, e80a7453f73c4bb2a6bcaac705d19736; no voters:
I20260812 06:54:55.644184 21526 raft_consensus.cc:2781] T 1ab31d852d934ff2966f5f1af100b466 P a1317386ad9141689a7b9f543db83e85 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 06:54:55.644285 21526 raft_consensus.cc:686] T 1ab31d852d934ff2966f5f1af100b466 P a1317386ad9141689a7b9f543db83e85 [term 1 LEADER]: Becoming Leader. State: Replica: a1317386ad9141689a7b9f543db83e85, State: Running, Role: LEADER
I20260812 06:54:55.644498 21526 consensus_queue.cc:237] T 1ab31d852d934ff2966f5f1af100b466 P a1317386ad9141689a7b9f543db83e85 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e80a7453f73c4bb2a6bcaac705d19736" member_type: VOTER last_known_addr { host: "127.19.52.65" port: 46259 } } peers { permanent_uuid: "a7a637fb5800477db43eb0b6621d165a" member_type: VOTER last_known_addr { host: "127.19.52.66" port: 43119 } } peers { permanent_uuid: "a1317386ad9141689a7b9f543db83e85" member_type: VOTER last_known_addr { host: "127.19.52.67" port: 45375 } }
I20260812 06:54:55.646349 20937 catalog_manager.cc:5495] T 1ab31d852d934ff2966f5f1af100b466 P a1317386ad9141689a7b9f543db83e85 reported cstate change: term changed from 0 to 1, leader changed from <none> to a1317386ad9141689a7b9f543db83e85 (127.19.52.67). New cstate: current_term: 1 leader_uuid: "a1317386ad9141689a7b9f543db83e85" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e80a7453f73c4bb2a6bcaac705d19736" member_type: VOTER last_known_addr { host: "127.19.52.65" port: 46259 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a7a637fb5800477db43eb0b6621d165a" member_type: VOTER last_known_addr { host: "127.19.52.66" port: 43119 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a1317386ad9141689a7b9f543db83e85" member_type: VOTER last_known_addr { host: "127.19.52.67" port: 45375 } health_report { overall_health: HEALTHY } } }
Aug 12 06:54:55 dist-test-slave-65mx krb5kdc[20884](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517694, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.52.126@KRBTEST.COM for kudu/127.19.52.126@KRBTEST.COM
I20260812 06:54:55.695362 21553 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:54:55.686552 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.126:34419 (local address 127.0.0.1:57052)
0812 06:54:55.686937 (+ 385us) negotiation.cc:107] Waiting for socket to connect
0812 06:54:55.686955 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0812 06:54:55.687108 (+ 153us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:54:55.687380 (+ 272us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:54:55.687382 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:54:55.687859 (+ 477us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:54:55.687958 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:54:55.687964 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:54:55.688804 (+ 840us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:54:55.688809 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:54:55.689413 (+ 604us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:54:55.689422 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:54:55.689881 (+ 459us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:54:55.689884 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:54:55.689963 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:54:55.690560 (+ 597us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 06:54:55.690572 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:54:55.692134 (+ 1562us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 06:54:55.693904 (+ 1770us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:54:55.693913 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:54:55.693918 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:54:55.694157 (+ 239us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:54:55.694420 (+ 263us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:54:55.694422 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:54:55.694423 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:54:55.694530 (+ 107us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:54:55.694879 (+ 349us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 06:54:55.694885 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 06:54:55.695007 (+ 122us) client_negotiation.cc:769] Sending connection context
0812 06:54:55.695105 (+ 98us) client_negotiation.cc:240] Negotiation successful
0812 06:54:55.695221 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":259,"thread_start_us":135,"threads_started":1}
I20260812 06:54:55.695497 21552 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:54:55.686578 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57052 (local address 127.19.52.126:34419)
0812 06:54:55.686927 (+ 349us) server_negotiation.cc:206] Beginning negotiation
0812 06:54:55.686937 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0812 06:54:55.687043 (+ 106us) server_negotiation.cc:407] Connection header received
0812 06:54:55.687210 (+ 167us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:54:55.687213 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:54:55.687260 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:54:55.687360 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:54:55.688081 (+ 721us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:54:55.688695 (+ 614us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:54:55.689529 (+ 834us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:54:55.689762 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:54:55.689877 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:54:55.692296 (+ 2419us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:54:55.692299 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:54:55.692302 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:54:55.692324 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:54:55.693795 (+ 1471us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:54:55.694244 (+ 449us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:54:55.694246 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:54:55.694247 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:54:55.694321 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:54:55.694611 (+ 290us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:54:55.694613 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:54:55.694614 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:54:55.694787 (+ 173us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:54:55.694863 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0812 06:54:55.695197 (+ 334us) server_negotiation.cc:299] Negotiation successful
0812 06:54:55.695299 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":5,"server-negotiator.queue_time_us":298,"thread_start_us":128,"threads_started":1}
I20260812 06:54:55.699580 20937 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.19.52.126@KRBTEST.COM'} at 127.0.0.1:57052:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260812 06:54:55.700309 20937 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 06:54:55.705790 21259 tablet_service.cc:1467] Processing CreateTablet for tablet 6dd862abf95f4fb496e5924d924776e4 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=b682cbb39c0443d798df0f39a19ca2aa]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260812 06:54:55.705920 21418 tablet_service.cc:1467] Processing CreateTablet for tablet 6dd862abf95f4fb496e5924d924776e4 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=b682cbb39c0443d798df0f39a19ca2aa]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260812 06:54:55.706293 21259 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6dd862abf95f4fb496e5924d924776e4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 06:54:55.706318 21418 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6dd862abf95f4fb496e5924d924776e4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 06:54:55.705790 21096 tablet_service.cc:1467] Processing CreateTablet for tablet 6dd862abf95f4fb496e5924d924776e4 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=b682cbb39c0443d798df0f39a19ca2aa]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260812 06:54:55.707973 21096 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6dd862abf95f4fb496e5924d924776e4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 06:54:55.709049 21515 tablet_bootstrap.cc:492] T 6dd862abf95f4fb496e5924d924776e4 P a1317386ad9141689a7b9f543db83e85: Bootstrap starting.
I20260812 06:54:55.709154 21518 tablet_bootstrap.cc:492] T 6dd862abf95f4fb496e5924d924776e4 P a7a637fb5800477db43eb0b6621d165a: Bootstrap starting.
I20260812 06:54:55.709863 21515 tablet_bootstrap.cc:654] T 6dd862abf95f4fb496e5924d924776e4 P a1317386ad9141689a7b9f543db83e85: Neither blocks nor log segments found. Creating new log.
I20260812 06:54:55.710014 21518 tablet_bootstrap.cc:654] T 6dd862abf95f4fb496e5924d924776e4 P a7a637fb5800477db43eb0b6621d165a: Neither blocks nor log segments found. Creating new log.
I20260812 06:54:55.711009 21515 tablet_bootstrap.cc:492] T 6dd862abf95f4fb496e5924d924776e4 P a1317386ad9141689a7b9f543db83e85: No bootstrap required, opened a new log
I20260812 06:54:55.711120 21515 ts_tablet_manager.cc:1397] T 6dd862abf95f4fb496e5924d924776e4 P a1317386ad9141689a7b9f543db83e85: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260812 06:54:55.711472 21515 raft_consensus.cc:348] T 6dd862abf95f4fb496e5924d924776e4 P a1317386ad9141689a7b9f543db83e85 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e80a7453f73c4bb2a6bcaac705d19736" member_type: VOTER last_known_addr { host: "127.19.52.65" port: 46259 } } peers { permanent_uuid: "a7a637fb5800477db43eb0b6621d165a" member_type: VOTER last_known_addr { host: "127.19.52.66" port: 43119 } } peers { permanent_uuid: "a1317386ad9141689a7b9f543db83e85" member_type: VOTER last_known_addr { host: "127.19.52.67" port: 45375 } }
I20260812 06:54:55.711592 21515 raft_consensus.cc:374] T 6dd862abf95f4fb496e5924d924776e4 P a1317386ad9141689a7b9f543db83e85 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 06:54:55.711637 21515 raft_consensus.cc:729] T 6dd862abf95f4fb496e5924d924776e4 P a1317386ad9141689a7b9f543db83e85 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a1317386ad9141689a7b9f543db83e85, State: Initialized, Role: FOLLOWER
I20260812 06:54:55.711769 21515 consensus_queue.cc:260] T 6dd862abf95f4fb496e5924d924776e4 P a1317386ad9141689a7b9f543db83e85 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e80a7453f73c4bb2a6bcaac705d19736" member_type: VOTER last_known_addr { host: "127.19.52.65" port: 46259 } } peers { permanent_uuid: "a7a637fb5800477db43eb0b6621d165a" member_type: VOTER last_known_addr { host: "127.19.52.66" port: 43119 } } peers { permanent_uuid: "a1317386ad9141689a7b9f543db83e85" member_type: VOTER last_known_addr { host: "127.19.52.67" port: 45375 } }
I20260812 06:54:55.712076 21515 ts_tablet_manager.cc:1428] T 6dd862abf95f4fb496e5924d924776e4 P a1317386ad9141689a7b9f543db83e85: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260812 06:54:55.712074 21526 tablet_replica.cc:439] T 6dd862abf95f4fb496e5924d924776e4 P a1317386ad9141689a7b9f543db83e85: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e80a7453f73c4bb2a6bcaac705d19736" member_type: VOTER last_known_addr { host: "127.19.52.65" port: 46259 } } peers { permanent_uuid: "a7a637fb5800477db43eb0b6621d165a" member_type: VOTER last_known_addr { host: "127.19.52.66" port: 43119 } } peers { permanent_uuid: "a1317386ad9141689a7b9f543db83e85" member_type: VOTER last_known_addr { host: "127.19.52.67" port: 45375 } } }
I20260812 06:54:55.712339 21526 tablet_replica.cc:442] T 6dd862abf95f4fb496e5924d924776e4 P a1317386ad9141689a7b9f543db83e85: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 06:54:55.712911 21517 tablet_bootstrap.cc:492] T 6dd862abf95f4fb496e5924d924776e4 P e80a7453f73c4bb2a6bcaac705d19736: Bootstrap starting.
I20260812 06:54:55.713094 21518 tablet_bootstrap.cc:492] T 6dd862abf95f4fb496e5924d924776e4 P a7a637fb5800477db43eb0b6621d165a: No bootstrap required, opened a new log
I20260812 06:54:55.713184 21518 ts_tablet_manager.cc:1397] T 6dd862abf95f4fb496e5924d924776e4 P a7a637fb5800477db43eb0b6621d165a: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.000s
I20260812 06:54:55.713660 21518 raft_consensus.cc:348] T 6dd862abf95f4fb496e5924d924776e4 P a7a637fb5800477db43eb0b6621d165a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e80a7453f73c4bb2a6bcaac705d19736" member_type: VOTER last_known_addr { host: "127.19.52.65" port: 46259 } } peers { permanent_uuid: "a7a637fb5800477db43eb0b6621d165a" member_type: VOTER last_known_addr { host: "127.19.52.66" port: 43119 } } peers { permanent_uuid: "a1317386ad9141689a7b9f543db83e85" member_type: VOTER last_known_addr { host: "127.19.52.67" port: 45375 } }
I20260812 06:54:55.713773 21518 raft_consensus.cc:374] T 6dd862abf95f4fb496e5924d924776e4 P a7a637fb5800477db43eb0b6621d165a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 06:54:55.713810 21517 tablet_bootstrap.cc:654] T 6dd862abf95f4fb496e5924d924776e4 P e80a7453f73c4bb2a6bcaac705d19736: Neither blocks nor log segments found. Creating new log.
I20260812 06:54:55.713811 21518 raft_consensus.cc:729] T 6dd862abf95f4fb496e5924d924776e4 P a7a637fb5800477db43eb0b6621d165a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a7a637fb5800477db43eb0b6621d165a, State: Initialized, Role: FOLLOWER
I20260812 06:54:55.713940 21518 consensus_queue.cc:260] T 6dd862abf95f4fb496e5924d924776e4 P a7a637fb5800477db43eb0b6621d165a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e80a7453f73c4bb2a6bcaac705d19736" member_type: VOTER last_known_addr { host: "127.19.52.65" port: 46259 } } peers { permanent_uuid: "a7a637fb5800477db43eb0b6621d165a" member_type: VOTER last_known_addr { host: "127.19.52.66" port: 43119 } } peers { permanent_uuid: "a1317386ad9141689a7b9f543db83e85" member_type: VOTER last_known_addr { host: "127.19.52.67" port: 45375 } }
I20260812 06:54:55.714231 21524 tablet_replica.cc:439] T 6dd862abf95f4fb496e5924d924776e4 P a7a637fb5800477db43eb0b6621d165a: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e80a7453f73c4bb2a6bcaac705d19736" member_type: VOTER last_known_addr { host: "127.19.52.65" port: 46259 } } peers { permanent_uuid: "a7a637fb5800477db43eb0b6621d165a" member_type: VOTER last_known_addr { host: "127.19.52.66" port: 43119 } } peers { permanent_uuid: "a1317386ad9141689a7b9f543db83e85" member_type: VOTER last_known_addr { host: "127.19.52.67" port: 45375 } } }
I20260812 06:54:55.714394 21524 tablet_replica.cc:442] T 6dd862abf95f4fb496e5924d924776e4 P a7a637fb5800477db43eb0b6621d165a: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 06:54:55.714236 21518 ts_tablet_manager.cc:1428] T 6dd862abf95f4fb496e5924d924776e4 P a7a637fb5800477db43eb0b6621d165a: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260812 06:54:55.715605 21517 tablet_bootstrap.cc:492] T 6dd862abf95f4fb496e5924d924776e4 P e80a7453f73c4bb2a6bcaac705d19736: No bootstrap required, opened a new log
I20260812 06:54:55.715682 21517 ts_tablet_manager.cc:1397] T 6dd862abf95f4fb496e5924d924776e4 P e80a7453f73c4bb2a6bcaac705d19736: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.002s
I20260812 06:54:55.716182 21517 raft_consensus.cc:348] T 6dd862abf95f4fb496e5924d924776e4 P e80a7453f73c4bb2a6bcaac705d19736 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e80a7453f73c4bb2a6bcaac705d19736" member_type: VOTER last_known_addr { host: "127.19.52.65" port: 46259 } } peers { permanent_uuid: "a7a637fb5800477db43eb0b6621d165a" member_type: VOTER last_known_addr { host: "127.19.52.66" port: 43119 } } peers { permanent_uuid: "a1317386ad9141689a7b9f543db83e85" member_type: VOTER last_known_addr { host: "127.19.52.67" port: 45375 } }
I20260812 06:54:55.716331 21517 raft_consensus.cc:374] T 6dd862abf95f4fb496e5924d924776e4 P e80a7453f73c4bb2a6bcaac705d19736 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 06:54:55.716378 21517 raft_consensus.cc:729] T 6dd862abf95f4fb496e5924d924776e4 P e80a7453f73c4bb2a6bcaac705d19736 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e80a7453f73c4bb2a6bcaac705d19736, State: Initialized, Role: FOLLOWER
I20260812 06:54:55.716523 21517 consensus_queue.cc:260] T 6dd862abf95f4fb496e5924d924776e4 P e80a7453f73c4bb2a6bcaac705d19736 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e80a7453f73c4bb2a6bcaac705d19736" member_type: VOTER last_known_addr { host: "127.19.52.65" port: 46259 } } peers { permanent_uuid: "a7a637fb5800477db43eb0b6621d165a" member_type: VOTER last_known_addr { host: "127.19.52.66" port: 43119 } } peers { permanent_uuid: "a1317386ad9141689a7b9f543db83e85" member_type: VOTER last_known_addr { host: "127.19.52.67" port: 45375 } }
I20260812 06:54:55.716845 21517 ts_tablet_manager.cc:1428] T 6dd862abf95f4fb496e5924d924776e4 P e80a7453f73c4bb2a6bcaac705d19736: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260812 06:54:55.717016 21522 tablet_replica.cc:439] T 6dd862abf95f4fb496e5924d924776e4 P e80a7453f73c4bb2a6bcaac705d19736: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e80a7453f73c4bb2a6bcaac705d19736" member_type: VOTER last_known_addr { host: "127.19.52.65" port: 46259 } } peers { permanent_uuid: "a7a637fb5800477db43eb0b6621d165a" member_type: VOTER last_known_addr { host: "127.19.52.66" port: 43119 } } peers { permanent_uuid: "a1317386ad9141689a7b9f543db83e85" member_type: VOTER last_known_addr { host: "127.19.52.67" port: 45375 } } }
I20260812 06:54:55.717201 21522 tablet_replica.cc:442] T 6dd862abf95f4fb496e5924d924776e4 P e80a7453f73c4bb2a6bcaac705d19736: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 06:54:55.740061 21522 raft_consensus.cc:482] T 6dd862abf95f4fb496e5924d924776e4 P e80a7453f73c4bb2a6bcaac705d19736 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 06:54:55.740252 21522 raft_consensus.cc:504] T 6dd862abf95f4fb496e5924d924776e4 P e80a7453f73c4bb2a6bcaac705d19736 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e80a7453f73c4bb2a6bcaac705d19736" member_type: VOTER last_known_addr { host: "127.19.52.65" port: 46259 } } peers { permanent_uuid: "a7a637fb5800477db43eb0b6621d165a" member_type: VOTER last_known_addr { host: "127.19.52.66" port: 43119 } } peers { permanent_uuid: "a1317386ad9141689a7b9f543db83e85" member_type: VOTER last_known_addr { host: "127.19.52.67" port: 45375 } }
I20260812 06:54:55.741602 21522 leader_election.cc:288] T 6dd862abf95f4fb496e5924d924776e4 P e80a7453f73c4bb2a6bcaac705d19736 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a7a637fb5800477db43eb0b6621d165a (127.19.52.66:43119), a1317386ad9141689a7b9f543db83e85 (127.19.52.67:45375)
I20260812 06:54:55.746237 21510 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:54:55.741930 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.52.65:40245 (local address 127.19.52.66:43119)
0812 06:54:55.742100 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0812 06:54:55.742107 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 06:54:55.742402 (+ 295us) server_negotiation.cc:407] Connection header received
0812 06:54:55.742594 (+ 192us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:54:55.742597 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:54:55.742645 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:54:55.742738 (+ 93us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 06:54:55.743075 (+ 337us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:54:55.743622 (+ 547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:54:55.744914 (+ 1292us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:54:55.745501 (+ 587us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:54:55.745630 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:54:55.745687 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0812 06:54:55.746049 (+ 362us) server_negotiation.cc:299] Negotiation successful
0812 06:54:55.746125 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260812 06:54:55.746337 21513 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:54:55.742112 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.52.65:35323 (local address 127.19.52.67:45375)
0812 06:54:55.742254 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0812 06:54:55.742259 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 06:54:55.742476 (+ 217us) server_negotiation.cc:407] Connection header received
0812 06:54:55.742646 (+ 170us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:54:55.742649 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:54:55.742705 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:54:55.742797 (+ 92us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 06:54:55.743115 (+ 318us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:54:55.743610 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:54:55.744902 (+ 1292us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:54:55.745383 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:54:55.745504 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:54:55.745559 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0812 06:54:55.746098 (+ 539us) server_negotiation.cc:299] Negotiation successful
0812 06:54:55.746192 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260812 06:54:55.746621 21558 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:54:55.741813 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.66:43119 (local address 127.19.52.65:40245)
0812 06:54:55.742263 (+ 450us) negotiation.cc:107] Waiting for socket to connect
0812 06:54:55.742282 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0812 06:54:55.742455 (+ 173us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:54:55.742750 (+ 295us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:54:55.742753 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:54:55.742770 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 06:54:55.742940 (+ 170us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:54:55.742946 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:54:55.743766 (+ 820us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:54:55.743770 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:54:55.744768 (+ 998us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:54:55.744778 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:54:55.745744 (+ 966us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:54:55.745747 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:54:55.745844 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:54:55.745912 (+ 68us) client_negotiation.cc:769] Sending connection context
0812 06:54:55.746396 (+ 484us) client_negotiation.cc:240] Negotiation successful
0812 06:54:55.746477 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":382,"mutex_wait_us":90,"thread_start_us":162,"threads_started":1}
I20260812 06:54:55.746969 21557 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:54:55.741975 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.67:45375 (local address 127.19.52.65:35323)
0812 06:54:55.742353 (+ 378us) negotiation.cc:107] Waiting for socket to connect
0812 06:54:55.742365 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 06:54:55.742519 (+ 154us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:54:55.742804 (+ 285us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:54:55.742806 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:54:55.742823 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 06:54:55.742986 (+ 163us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:54:55.742992 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:54:55.743728 (+ 736us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:54:55.743732 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:54:55.744762 (+ 1030us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:54:55.744772 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:54:55.745776 (+ 1004us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:54:55.745780 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:54:55.745926 (+ 146us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:54:55.745954 (+ 28us) client_negotiation.cc:769] Sending connection context
0812 06:54:55.746494 (+ 540us) client_negotiation.cc:240] Negotiation successful
0812 06:54:55.746837 (+ 343us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":322,"mutex_wait_us":78,"thread_start_us":114,"threads_started":1}
I20260812 06:54:55.747133 21281 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6dd862abf95f4fb496e5924d924776e4" candidate_uuid: "e80a7453f73c4bb2a6bcaac705d19736" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a7a637fb5800477db43eb0b6621d165a" is_pre_election: true
I20260812 06:54:55.747282 21281 raft_consensus.cc:2454] T 6dd862abf95f4fb496e5924d924776e4 P a7a637fb5800477db43eb0b6621d165a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e80a7453f73c4bb2a6bcaac705d19736 in term 0.
I20260812 06:54:55.747581 21442 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6dd862abf95f4fb496e5924d924776e4" candidate_uuid: "e80a7453f73c4bb2a6bcaac705d19736" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a1317386ad9141689a7b9f543db83e85" is_pre_election: true
I20260812 06:54:55.747677 21043 leader_election.cc:302] T 6dd862abf95f4fb496e5924d924776e4 P e80a7453f73c4bb2a6bcaac705d19736 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a7a637fb5800477db43eb0b6621d165a, e80a7453f73c4bb2a6bcaac705d19736; no voters:
I20260812 06:54:55.747852 21442 raft_consensus.cc:2454] T 6dd862abf95f4fb496e5924d924776e4 P a1317386ad9141689a7b9f543db83e85 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e80a7453f73c4bb2a6bcaac705d19736 in term 0.
I20260812 06:54:55.747972 21522 raft_consensus.cc:2781] T 6dd862abf95f4fb496e5924d924776e4 P e80a7453f73c4bb2a6bcaac705d19736 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 06:54:55.748032 21522 raft_consensus.cc:482] T 6dd862abf95f4fb496e5924d924776e4 P e80a7453f73c4bb2a6bcaac705d19736 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 06:54:55.748083 21522 raft_consensus.cc:3037] T 6dd862abf95f4fb496e5924d924776e4 P e80a7453f73c4bb2a6bcaac705d19736 [term 0 FOLLOWER]: Advancing to term 1
I20260812 06:54:55.749329 21522 raft_consensus.cc:504] T 6dd862abf95f4fb496e5924d924776e4 P e80a7453f73c4bb2a6bcaac705d19736 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e80a7453f73c4bb2a6bcaac705d19736" member_type: VOTER last_known_addr { host: "127.19.52.65" port: 46259 } } peers { permanent_uuid: "a7a637fb5800477db43eb0b6621d165a" member_type: VOTER last_known_addr { host: "127.19.52.66" port: 43119 } } peers { permanent_uuid: "a1317386ad9141689a7b9f543db83e85" member_type: VOTER last_known_addr { host: "127.19.52.67" port: 45375 } }
I20260812 06:54:55.749752 21522 leader_election.cc:288] T 6dd862abf95f4fb496e5924d924776e4 P e80a7453f73c4bb2a6bcaac705d19736 [CANDIDATE]: Term 1 election: Requested vote from peers a7a637fb5800477db43eb0b6621d165a (127.19.52.66:43119), a1317386ad9141689a7b9f543db83e85 (127.19.52.67:45375)
I20260812 06:54:55.750077 21281 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6dd862abf95f4fb496e5924d924776e4" candidate_uuid: "e80a7453f73c4bb2a6bcaac705d19736" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a7a637fb5800477db43eb0b6621d165a"
I20260812 06:54:55.750195 21281 raft_consensus.cc:3037] T 6dd862abf95f4fb496e5924d924776e4 P a7a637fb5800477db43eb0b6621d165a [term 0 FOLLOWER]: Advancing to term 1
I20260812 06:54:55.750159 21442 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6dd862abf95f4fb496e5924d924776e4" candidate_uuid: "e80a7453f73c4bb2a6bcaac705d19736" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a1317386ad9141689a7b9f543db83e85"
I20260812 06:54:55.750288 21442 raft_consensus.cc:3037] T 6dd862abf95f4fb496e5924d924776e4 P a1317386ad9141689a7b9f543db83e85 [term 0 FOLLOWER]: Advancing to term 1
I20260812 06:54:55.751374 21281 raft_consensus.cc:2454] T 6dd862abf95f4fb496e5924d924776e4 P a7a637fb5800477db43eb0b6621d165a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e80a7453f73c4bb2a6bcaac705d19736 in term 1.
I20260812 06:54:55.751397 21442 raft_consensus.cc:2454] T 6dd862abf95f4fb496e5924d924776e4 P a1317386ad9141689a7b9f543db83e85 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e80a7453f73c4bb2a6bcaac705d19736 in term 1.
I20260812 06:54:55.751693 21043 leader_election.cc:302] T 6dd862abf95f4fb496e5924d924776e4 P e80a7453f73c4bb2a6bcaac705d19736 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a7a637fb5800477db43eb0b6621d165a, e80a7453f73c4bb2a6bcaac705d19736; no voters:
I20260812 06:54:55.751888 21522 raft_consensus.cc:2781] T 6dd862abf95f4fb496e5924d924776e4 P e80a7453f73c4bb2a6bcaac705d19736 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 06:54:55.752211 21522 raft_consensus.cc:686] T 6dd862abf95f4fb496e5924d924776e4 P e80a7453f73c4bb2a6bcaac705d19736 [term 1 LEADER]: Becoming Leader. State: Replica: e80a7453f73c4bb2a6bcaac705d19736, State: Running, Role: LEADER
I20260812 06:54:55.752516 21522 consensus_queue.cc:237] T 6dd862abf95f4fb496e5924d924776e4 P e80a7453f73c4bb2a6bcaac705d19736 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e80a7453f73c4bb2a6bcaac705d19736" member_type: VOTER last_known_addr { host: "127.19.52.65" port: 46259 } } peers { permanent_uuid: "a7a637fb5800477db43eb0b6621d165a" member_type: VOTER last_known_addr { host: "127.19.52.66" port: 43119 } } peers { permanent_uuid: "a1317386ad9141689a7b9f543db83e85" member_type: VOTER last_known_addr { host: "127.19.52.67" port: 45375 } }
I20260812 06:54:55.754962 21559 tablet_replica.cc:439] T 6dd862abf95f4fb496e5924d924776e4 P e80a7453f73c4bb2a6bcaac705d19736: TxnStatusTablet state changed. Reason: New leader e80a7453f73c4bb2a6bcaac705d19736. Latest consensus state: current_term: 1 leader_uuid: "e80a7453f73c4bb2a6bcaac705d19736" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e80a7453f73c4bb2a6bcaac705d19736" member_type: VOTER last_known_addr { host: "127.19.52.65" port: 46259 } } peers { permanent_uuid: "a7a637fb5800477db43eb0b6621d165a" member_type: VOTER last_known_addr { host: "127.19.52.66" port: 43119 } } peers { permanent_uuid: "a1317386ad9141689a7b9f543db83e85" member_type: VOTER last_known_addr { host: "127.19.52.67" port: 45375 } } }
I20260812 06:54:55.755106 21559 tablet_replica.cc:442] T 6dd862abf95f4fb496e5924d924776e4 P e80a7453f73c4bb2a6bcaac705d19736: This TxnStatusTablet replica's current role is: LEADER
I20260812 06:54:55.755467 21561 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260812 06:54:55.755693 21561 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260812 06:54:55.756201 20938 catalog_manager.cc:5495] T 6dd862abf95f4fb496e5924d924776e4 P e80a7453f73c4bb2a6bcaac705d19736 reported cstate change: term changed from 0 to 1, leader changed from <none> to e80a7453f73c4bb2a6bcaac705d19736 (127.19.52.65). New cstate: current_term: 1 leader_uuid: "e80a7453f73c4bb2a6bcaac705d19736" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e80a7453f73c4bb2a6bcaac705d19736" member_type: VOTER last_known_addr { host: "127.19.52.65" port: 46259 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a7a637fb5800477db43eb0b6621d165a" member_type: VOTER last_known_addr { host: "127.19.52.66" port: 43119 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a1317386ad9141689a7b9f543db83e85" member_type: VOTER last_known_addr { host: "127.19.52.67" port: 45375 } health_report { overall_health: UNKNOWN } } }
I20260812 06:54:55.779561 21553 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:54:55.774177 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.65:46259 (local address 127.0.0.1:33836)
0812 06:54:55.774347 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0812 06:54:55.774366 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0812 06:54:55.774571 (+ 205us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:54:55.775039 (+ 468us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:54:55.775042 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:54:55.775050 (+ 8us) client_negotiation.cc:189] Negotiated authn=TOKEN
0812 06:54:55.775165 (+ 115us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:54:55.775171 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:54:55.776346 (+ 1175us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:54:55.776349 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:54:55.777681 (+ 1332us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:54:55.777695 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:54:55.778304 (+ 609us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:54:55.778307 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:54:55.778385 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:54:55.778456 (+ 71us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0812 06:54:55.779270 (+ 814us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0812 06:54:55.779287 (+ 17us) client_negotiation.cc:769] Sending connection context
0812 06:54:55.779352 (+ 65us) client_negotiation.cc:240] Negotiation successful
0812 06:54:55.779434 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":56}
I20260812 06:54:55.779603 21514 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:54:55.774431 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33836 (local address 127.19.52.65:46259)
0812 06:54:55.774656 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0812 06:54:55.774660 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 06:54:55.774676 (+ 16us) server_negotiation.cc:407] Connection header received
0812 06:54:55.774757 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:54:55.774760 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:54:55.774828 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:54:55.774945 (+ 117us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 06:54:55.775289 (+ 344us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:54:55.776184 (+ 895us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:54:55.777821 (+ 1637us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:54:55.778168 (+ 347us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:54:55.778276 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:54:55.778614 (+ 338us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 06:54:55.779151 (+ 537us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 06:54:55.779268 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0812 06:54:55.779396 (+ 128us) server_negotiation.cc:299] Negotiation successful
0812 06:54:55.779464 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260812 06:54:55.783056 21281 raft_consensus.cc:1261] T 6dd862abf95f4fb496e5924d924776e4 P a7a637fb5800477db43eb0b6621d165a [term 1 FOLLOWER]: Refusing update from remote peer e80a7453f73c4bb2a6bcaac705d19736: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 06:54:55.783259 21442 raft_consensus.cc:1261] T 6dd862abf95f4fb496e5924d924776e4 P a1317386ad9141689a7b9f543db83e85 [term 1 FOLLOWER]: Refusing update from remote peer e80a7453f73c4bb2a6bcaac705d19736: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 06:54:55.783618 21522 consensus_queue.cc:1035] T 6dd862abf95f4fb496e5924d924776e4 P e80a7453f73c4bb2a6bcaac705d19736 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a7a637fb5800477db43eb0b6621d165a" member_type: VOTER last_known_addr { host: "127.19.52.66" port: 43119 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 06:54:55.783887 21559 consensus_queue.cc:1035] T 6dd862abf95f4fb496e5924d924776e4 P e80a7453f73c4bb2a6bcaac705d19736 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a1317386ad9141689a7b9f543db83e85" member_type: VOTER last_known_addr { host: "127.19.52.67" port: 45375 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 06:54:55.785392 21524 tablet_replica.cc:439] T 6dd862abf95f4fb496e5924d924776e4 P a7a637fb5800477db43eb0b6621d165a: TxnStatusTablet state changed. Reason: New leader e80a7453f73c4bb2a6bcaac705d19736. Latest consensus state: current_term: 1 leader_uuid: "e80a7453f73c4bb2a6bcaac705d19736" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e80a7453f73c4bb2a6bcaac705d19736" member_type: VOTER last_known_addr { host: "127.19.52.65" port: 46259 } } peers { permanent_uuid: "a7a637fb5800477db43eb0b6621d165a" member_type: VOTER last_known_addr { host: "127.19.52.66" port: 43119 } } peers { permanent_uuid: "a1317386ad9141689a7b9f543db83e85" member_type: VOTER last_known_addr { host: "127.19.52.67" port: 45375 } } }
I20260812 06:54:55.785537 21524 tablet_replica.cc:442] T 6dd862abf95f4fb496e5924d924776e4 P a7a637fb5800477db43eb0b6621d165a: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 06:54:55.785682 21526 tablet_replica.cc:439] T 6dd862abf95f4fb496e5924d924776e4 P a1317386ad9141689a7b9f543db83e85: TxnStatusTablet state changed. Reason: New leader e80a7453f73c4bb2a6bcaac705d19736. Latest consensus state: current_term: 1 leader_uuid: "e80a7453f73c4bb2a6bcaac705d19736" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e80a7453f73c4bb2a6bcaac705d19736" member_type: VOTER last_known_addr { host: "127.19.52.65" port: 46259 } } peers { permanent_uuid: "a7a637fb5800477db43eb0b6621d165a" member_type: VOTER last_known_addr { host: "127.19.52.66" port: 43119 } } peers { permanent_uuid: "a1317386ad9141689a7b9f543db83e85" member_type: VOTER last_known_addr { host: "127.19.52.67" port: 45375 } } }
I20260812 06:54:55.785799 21526 tablet_replica.cc:442] T 6dd862abf95f4fb496e5924d924776e4 P a1317386ad9141689a7b9f543db83e85: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 06:54:55.788522 21526 tablet_replica.cc:439] T 6dd862abf95f4fb496e5924d924776e4 P a1317386ad9141689a7b9f543db83e85: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "e80a7453f73c4bb2a6bcaac705d19736" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e80a7453f73c4bb2a6bcaac705d19736" member_type: VOTER last_known_addr { host: "127.19.52.65" port: 46259 } } peers { permanent_uuid: "a7a637fb5800477db43eb0b6621d165a" member_type: VOTER last_known_addr { host: "127.19.52.66" port: 43119 } } peers { permanent_uuid: "a1317386ad9141689a7b9f543db83e85" member_type: VOTER last_known_addr { host: "127.19.52.67" port: 45375 } } }
I20260812 06:54:55.788676 21526 tablet_replica.cc:442] T 6dd862abf95f4fb496e5924d924776e4 P a1317386ad9141689a7b9f543db83e85: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 06:54:55.788645 21529 mvcc.cc:205] Tried to move back new op lower bound from 7317576481921859584 to 7317576481805680640. Current Snapshot: MvccSnapshot[applied={T|T < 7317576481921859584}]
I20260812 06:54:55.789705 21559 tablet_replica.cc:439] T 6dd862abf95f4fb496e5924d924776e4 P e80a7453f73c4bb2a6bcaac705d19736: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "e80a7453f73c4bb2a6bcaac705d19736" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e80a7453f73c4bb2a6bcaac705d19736" member_type: VOTER last_known_addr { host: "127.19.52.65" port: 46259 } } peers { permanent_uuid: "a7a637fb5800477db43eb0b6621d165a" member_type: VOTER last_known_addr { host: "127.19.52.66" port: 43119 } } peers { permanent_uuid: "a1317386ad9141689a7b9f543db83e85" member_type: VOTER last_known_addr { host: "127.19.52.67" port: 45375 } } }
I20260812 06:54:55.789840 21559 tablet_replica.cc:442] T 6dd862abf95f4fb496e5924d924776e4 P e80a7453f73c4bb2a6bcaac705d19736: This TxnStatusTablet replica's current role is: LEADER
I20260812 06:54:55.790191 21559 tablet_replica.cc:439] T 6dd862abf95f4fb496e5924d924776e4 P e80a7453f73c4bb2a6bcaac705d19736: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "e80a7453f73c4bb2a6bcaac705d19736" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e80a7453f73c4bb2a6bcaac705d19736" member_type: VOTER last_known_addr { host: "127.19.52.65" port: 46259 } } peers { permanent_uuid: "a7a637fb5800477db43eb0b6621d165a" member_type: VOTER last_known_addr { host: "127.19.52.66" port: 43119 } } peers { permanent_uuid: "a1317386ad9141689a7b9f543db83e85" member_type: VOTER last_known_addr { host: "127.19.52.67" port: 45375 } } }
I20260812 06:54:55.790285 21559 tablet_replica.cc:442] T 6dd862abf95f4fb496e5924d924776e4 P e80a7453f73c4bb2a6bcaac705d19736: This TxnStatusTablet replica's current role is: LEADER
I20260812 06:54:55.791260 21524 tablet_replica.cc:439] T 6dd862abf95f4fb496e5924d924776e4 P a7a637fb5800477db43eb0b6621d165a: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "e80a7453f73c4bb2a6bcaac705d19736" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e80a7453f73c4bb2a6bcaac705d19736" member_type: VOTER last_known_addr { host: "127.19.52.65" port: 46259 } } peers { permanent_uuid: "a7a637fb5800477db43eb0b6621d165a" member_type: VOTER last_known_addr { host: "127.19.52.66" port: 43119 } } peers { permanent_uuid: "a1317386ad9141689a7b9f543db83e85" member_type: VOTER last_known_addr { host: "127.19.52.67" port: 45375 } } }
I20260812 06:54:55.791373 21524 tablet_replica.cc:442] T 6dd862abf95f4fb496e5924d924776e4 P a7a637fb5800477db43eb0b6621d165a: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 06:54:55.791471 21532 mvcc.cc:205] Tried to move back new op lower bound from 7317576481921859584 to 7317576481805680640. Current Snapshot: MvccSnapshot[applied={T|T < 7317576481921859584 or (T in {7317576481921859584})}]
I20260812 06:54:55.805660 21514 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:54:55.801934 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.52.67:48321 (local address 127.19.52.65:46259)
0812 06:54:55.802099 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0812 06:54:55.802103 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 06:54:55.802257 (+ 154us) server_negotiation.cc:407] Connection header received
0812 06:54:55.802461 (+ 204us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:54:55.802465 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:54:55.802524 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:54:55.802616 (+ 92us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 06:54:55.802873 (+ 257us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:54:55.803393 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:54:55.804282 (+ 889us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:54:55.804662 (+ 380us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:54:55.804777 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:54:55.805113 (+ 336us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 06:54:55.805254 (+ 141us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 06:54:55.805346 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0812 06:54:55.805441 (+ 95us) server_negotiation.cc:299] Negotiation successful
0812 06:54:55.805498 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260812 06:54:55.805697 21571 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:54:55.801788 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.65:46259 (local address 127.19.52.67:48321)
0812 06:54:55.802162 (+ 374us) negotiation.cc:107] Waiting for socket to connect
0812 06:54:55.802175 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 06:54:55.802321 (+ 146us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:54:55.802629 (+ 308us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:54:55.802632 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:54:55.802640 (+ 8us) client_negotiation.cc:189] Negotiated authn=TOKEN
0812 06:54:55.802762 (+ 122us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:54:55.802767 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:54:55.803520 (+ 753us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:54:55.803524 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:54:55.804156 (+ 632us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:54:55.804164 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:54:55.804784 (+ 620us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:54:55.804788 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:54:55.804879 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:54:55.804945 (+ 66us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0812 06:54:55.805350 (+ 405us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0812 06:54:55.805360 (+ 10us) client_negotiation.cc:769] Sending connection context
0812 06:54:55.805424 (+ 64us) client_negotiation.cc:240] Negotiation successful
0812 06:54:55.805523 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":259,"thread_start_us":128,"threads_started":1}
I20260812 06:54:55.814247 21118 raft_consensus.cc:1261] T 1ab31d852d934ff2966f5f1af100b466 P e80a7453f73c4bb2a6bcaac705d19736 [term 1 FOLLOWER]: Refusing update from remote peer a1317386ad9141689a7b9f543db83e85: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 06:54:55.814280 21281 raft_consensus.cc:1261] T 1ab31d852d934ff2966f5f1af100b466 P a7a637fb5800477db43eb0b6621d165a [term 1 FOLLOWER]: Refusing update from remote peer a1317386ad9141689a7b9f543db83e85: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 06:54:55.814729 21526 consensus_queue.cc:1035] T 1ab31d852d934ff2966f5f1af100b466 P a1317386ad9141689a7b9f543db83e85 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e80a7453f73c4bb2a6bcaac705d19736" member_type: VOTER last_known_addr { host: "127.19.52.65" port: 46259 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 06:54:55.814895 21523 consensus_queue.cc:1035] T 1ab31d852d934ff2966f5f1af100b466 P a1317386ad9141689a7b9f543db83e85 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a7a637fb5800477db43eb0b6621d165a" member_type: VOTER last_known_addr { host: "127.19.52.66" port: 43119 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 06:54:55.840998 21513 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:54:55.835779 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.52.65:42129 (local address 127.19.52.67:45375)
0812 06:54:55.835907 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0812 06:54:55.835913 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 06:54:55.836871 (+ 958us) server_negotiation.cc:407] Connection header received
0812 06:54:55.837081 (+ 210us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:54:55.837086 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:54:55.837146 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:54:55.837195 (+ 49us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 06:54:55.837597 (+ 402us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:54:55.838108 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:54:55.839613 (+ 1505us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:54:55.839963 (+ 350us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:54:55.840044 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:54:55.840506 (+ 462us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 06:54:55.840635 (+ 129us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 06:54:55.840706 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0812 06:54:55.840825 (+ 119us) server_negotiation.cc:299] Negotiation successful
0812 06:54:55.840878 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260812 06:54:55.841076 21576 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:54:55.835714 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.67:45375 (local address 127.19.52.65:42129)
0812 06:54:55.836775 (+ 1061us) negotiation.cc:107] Waiting for socket to connect
0812 06:54:55.836792 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0812 06:54:55.836937 (+ 145us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:54:55.837335 (+ 398us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:54:55.837338 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:54:55.837346 (+ 8us) client_negotiation.cc:189] Negotiated authn=TOKEN
0812 06:54:55.837490 (+ 144us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:54:55.837496 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:54:55.838221 (+ 725us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:54:55.838224 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:54:55.839469 (+ 1245us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:54:55.839482 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:54:55.840160 (+ 678us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:54:55.840163 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:54:55.840264 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:54:55.840318 (+ 54us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0812 06:54:55.840732 (+ 414us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0812 06:54:55.840739 (+ 7us) client_negotiation.cc:769] Sending connection context
0812 06:54:55.840817 (+ 78us) client_negotiation.cc:240] Negotiation successful
0812 06:54:55.840902 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":926,"thread_start_us":122,"threads_started":1}
I20260812 06:54:55.869447 20938 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:33374:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 06:54:55.869618 20938 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:33374:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 06:54:55.871863 20938 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 71ba5b0b680f4403a6860d61d029a0d9: Sending DeleteTablet for 3 replicas of tablet 1ab31d852d934ff2966f5f1af100b466
I20260812 06:54:55.872583 21418 tablet_service.cc:1514] Processing DeleteTablet for tablet 1ab31d852d934ff2966f5f1af100b466 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 06:54:55 UTC) from {username='kudu', principal='kudu/127.19.52.126@KRBTEST.COM'} at 127.0.0.1:53288
I20260812 06:54:55.872637 21259 tablet_service.cc:1514] Processing DeleteTablet for tablet 1ab31d852d934ff2966f5f1af100b466 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 06:54:55 UTC) from {username='kudu', principal='kudu/127.19.52.126@KRBTEST.COM'} at 127.0.0.1:40262
I20260812 06:54:55.872812 21539 tablet_replica.cc:330] T 1ab31d852d934ff2966f5f1af100b466 P a1317386ad9141689a7b9f543db83e85: stopping tablet replica
I20260812 06:54:55.872862 21541 tablet_replica.cc:330] T 1ab31d852d934ff2966f5f1af100b466 P a7a637fb5800477db43eb0b6621d165a: stopping tablet replica
I20260812 06:54:55.872972 21539 raft_consensus.cc:2229] T 1ab31d852d934ff2966f5f1af100b466 P a1317386ad9141689a7b9f543db83e85 [term 1 LEADER]: Raft consensus shutting down.
I20260812 06:54:55.872988 21541 raft_consensus.cc:2229] T 1ab31d852d934ff2966f5f1af100b466 P a7a637fb5800477db43eb0b6621d165a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 06:54:55.873112 21541 raft_consensus.cc:2258] T 1ab31d852d934ff2966f5f1af100b466 P a7a637fb5800477db43eb0b6621d165a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 06:54:55.873196 21539 raft_consensus.cc:2258] T 1ab31d852d934ff2966f5f1af100b466 P a1317386ad9141689a7b9f543db83e85 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 06:54:55.873303 21096 tablet_service.cc:1514] Processing DeleteTablet for tablet 1ab31d852d934ff2966f5f1af100b466 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 06:54:55 UTC) from {username='kudu', principal='kudu/127.19.52.126@KRBTEST.COM'} at 127.0.0.1:54764
I20260812 06:54:55.873522 21538 tablet_replica.cc:330] T 1ab31d852d934ff2966f5f1af100b466 P e80a7453f73c4bb2a6bcaac705d19736: stopping tablet replica
I20260812 06:54:55.873636 21538 raft_consensus.cc:2229] T 1ab31d852d934ff2966f5f1af100b466 P e80a7453f73c4bb2a6bcaac705d19736 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 06:54:55.873757 21538 raft_consensus.cc:2258] T 1ab31d852d934ff2966f5f1af100b466 P e80a7453f73c4bb2a6bcaac705d19736 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 06:54:55.874068 21539 ts_tablet_manager.cc:1905] T 1ab31d852d934ff2966f5f1af100b466 P a1317386ad9141689a7b9f543db83e85: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 06:54:55.874068 21541 ts_tablet_manager.cc:1905] T 1ab31d852d934ff2966f5f1af100b466 P a7a637fb5800477db43eb0b6621d165a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 06:54:55.877086 21539 ts_tablet_manager.cc:1918] T 1ab31d852d934ff2966f5f1af100b466 P a1317386ad9141689a7b9f543db83e85: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260812 06:54:55.877086 21541 ts_tablet_manager.cc:1918] T 1ab31d852d934ff2966f5f1af100b466 P a7a637fb5800477db43eb0b6621d165a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260812 06:54:55.877198 21539 log.cc:1196] T 1ab31d852d934ff2966f5f1af100b466 P a1317386ad9141689a7b9f543db83e85: Deleting WAL directory at /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786517687384214-19665-0/minicluster-data/ts-2/wal/wals/1ab31d852d934ff2966f5f1af100b466
I20260812 06:54:55.877198 21541 log.cc:1196] T 1ab31d852d934ff2966f5f1af100b466 P a7a637fb5800477db43eb0b6621d165a: Deleting WAL directory at /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786517687384214-19665-0/minicluster-data/ts-1/wal/wals/1ab31d852d934ff2966f5f1af100b466
I20260812 06:54:55.877519 21539 ts_tablet_manager.cc:1939] T 1ab31d852d934ff2966f5f1af100b466 P a1317386ad9141689a7b9f543db83e85: Deleting consensus metadata
I20260812 06:54:55.877519 21541 ts_tablet_manager.cc:1939] T 1ab31d852d934ff2966f5f1af100b466 P a7a637fb5800477db43eb0b6621d165a: Deleting consensus metadata
I20260812 06:54:55.878401 20919 catalog_manager.cc:4841] TS a1317386ad9141689a7b9f543db83e85 (127.19.52.67:45375): tablet 1ab31d852d934ff2966f5f1af100b466 (table test-table [id=71e174c7534e42cdab5f5d67174d34c6]) successfully deleted
I20260812 06:54:55.878645 20921 catalog_manager.cc:4841] TS a7a637fb5800477db43eb0b6621d165a (127.19.52.66:43119): tablet 1ab31d852d934ff2966f5f1af100b466 (table test-table [id=71e174c7534e42cdab5f5d67174d34c6]) successfully deleted
I20260812 06:54:55.879120 21538 ts_tablet_manager.cc:1905] T 1ab31d852d934ff2966f5f1af100b466 P e80a7453f73c4bb2a6bcaac705d19736: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 06:54:55.882200 21538 ts_tablet_manager.cc:1918] T 1ab31d852d934ff2966f5f1af100b466 P e80a7453f73c4bb2a6bcaac705d19736: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260812 06:54:55.882303 21538 log.cc:1196] T 1ab31d852d934ff2966f5f1af100b466 P e80a7453f73c4bb2a6bcaac705d19736: Deleting WAL directory at /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786517687384214-19665-0/minicluster-data/ts-0/wal/wals/1ab31d852d934ff2966f5f1af100b466
I20260812 06:54:55.882570 21538 ts_tablet_manager.cc:1939] T 1ab31d852d934ff2966f5f1af100b466 P e80a7453f73c4bb2a6bcaac705d19736: Deleting consensus metadata
I20260812 06:54:55.883665 20921 catalog_manager.cc:4841] TS e80a7453f73c4bb2a6bcaac705d19736 (127.19.52.65:46259): tablet 1ab31d852d934ff2966f5f1af100b466 (table test-table [id=71e174c7534e42cdab5f5d67174d34c6]) successfully deleted
Aug 12 06:54:55 dist-test-slave-65mx krb5kdc[20884](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517695, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.19.52.65@KRBTEST.COM
I20260812 06:54:55.890830 21514 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:54:55.879719 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33844 (local address 127.19.52.65:46259)
0812 06:54:55.879857 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0812 06:54:55.879864 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 06:54:55.880136 (+ 272us) server_negotiation.cc:407] Connection header received
0812 06:54:55.880249 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:54:55.880252 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:54:55.880308 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:54:55.880399 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:54:55.881312 (+ 913us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:54:55.882193 (+ 881us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:54:55.883417 (+ 1224us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:54:55.883736 (+ 319us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:54:55.883862 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:54:55.886839 (+ 2977us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:54:55.886852 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:54:55.886870 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:54:55.886904 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:54:55.888837 (+ 1933us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:54:55.889365 (+ 528us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:54:55.889375 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:54:55.889376 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:54:55.889481 (+ 105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:54:55.889778 (+ 297us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:54:55.889781 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:54:55.889783 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:54:55.890067 (+ 284us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:54:55.890342 (+ 275us) server_negotiation.cc:1035] Waiting for connection context
0812 06:54:55.890490 (+ 148us) server_negotiation.cc:299] Negotiation successful
0812 06:54:55.890629 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
W20260812 06:54:55.891619 21164 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.SetFlag from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:33844
I20260812 06:54:55.903213 21552 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:54:55.893607 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57060 (local address 127.19.52.126:34419)
0812 06:54:55.893751 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0812 06:54:55.893756 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 06:54:55.893998 (+ 242us) server_negotiation.cc:407] Connection header received
0812 06:54:55.894051 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:54:55.894054 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:54:55.894100 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:54:55.894166 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:54:55.895071 (+ 905us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:54:55.895636 (+ 565us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:54:55.896819 (+ 1183us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:54:55.897117 (+ 298us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:54:55.897225 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:54:55.899039 (+ 1814us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:54:55.899043 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:54:55.899045 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:54:55.899075 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:54:55.900995 (+ 1920us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:54:55.901534 (+ 539us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:54:55.901537 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:54:55.901538 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:54:55.901633 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:54:55.901990 (+ 357us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:54:55.901993 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:54:55.901995 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:54:55.902217 (+ 222us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:54:55.902340 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0812 06:54:55.902656 (+ 316us) server_negotiation.cc:299] Negotiation successful
0812 06:54:55.902797 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
W20260812 06:54:55.903807 20938 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:57060
I20260812 06:54:55.905359 19665 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu with pid 21020
I20260812 06:54:55.915287 19665 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu with pid 21181
I20260812 06:54:55.924844 19665 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu with pid 21343
I20260812 06:54:55.934813 19665 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu with pid 20899
2026-08-12T06:54:55Z chronyd exiting
[ OK ] SecurityITest.SmokeTestAsAuthorizedUser (3216 ms)
[ RUN ] SecurityITest.TxnSmokeWithDifferentUserTypes
Loading random data
Initializing database '/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786517687384214-19665-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 06:54:56 dist-test-slave-65mx krb5kdc[21585](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 06:54:56 dist-test-slave-65mx krb5kdc[21585](info): set up 2 sockets
Aug 12 06:54:56 dist-test-slave-65mx krb5kdc[21585](info): commencing operation
krb5kdc: starting...
W20260812 06:54:57.053654 19665 mini_kdc.cc:121] Time spent starting KDC: real 1.091s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786517687384214-19665-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786517687384214-19665-0/krb5kdc/test-user.keytab.
Aug 12 06:54:57 dist-test-slave-65mx krb5kdc[21585](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517697, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T06:54:57Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T06:54:57Z Disabled control of system clock
WARNING: no policy specified for kudu/127.19.52.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.52.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.52.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786517687384214-19665-0/krb5kdc/kudu_127.19.52.126.keytab.
Entry for principal kudu/127.19.52.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786517687384214-19665-0/krb5kdc/kudu_127.19.52.126.keytab.
I20260812 06:54:57.200915 19665 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu
/tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786517687384214-19665-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786517687384214-19665-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786517687384214-19665-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786517687384214-19665-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.52.126:43141
--webserver_interface=127.19.52.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786517687384214-19665-0/krb5kdc/kudu_127.19.52.126.keytab
--principal=kudu/127.19.52.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.52.84:42943
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.19.52.126:43141
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786517687384214-19665-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786517687384214-19665-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786517687384214-19665-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 06:54:57.318745 21600 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:54:57.319026 21600 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:54:57.319068 21600 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 06:54:57.323136 21600 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 06:54:57.323210 21600 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260812 06:54:57.323228 21600 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 06:54:57.323246 21600 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 06:54:57.323295 21600 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 06:54:57.323335 21600 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 06:54:57.328860 21600 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.52.84:42943
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786517687384214-19665-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786517687384214-19665-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.19.52.126:43141
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786517687384214-19665-0/krb5kdc/kudu_127.19.52.126.keytab
--principal=kudu/127.19.52.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.52.126:43141
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786517687384214-19665-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.19.52.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21600
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786517687384214-19665-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 06:15:54 UTC on ccycloud.quasar-rnfzce.root.comops.site
build id 4581989
I20260812 06:54:57.330019 21600 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:54:57.331089 21600 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 06:54:57.340142 21605 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:54:57.340065 21608 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:54:57.340065 21606 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:54:57.340626 21600 server_base.cc:1034] running on GCE node
I20260812 06:54:57.341212 21600 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:54:57.342408 21600 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:54:57.343607 21600 hybrid_clock.cc:648] HybridClock initialized: now 1786517697343588 us; error 32 us; skew 500 ppm
Aug 12 06:54:57 dist-test-slave-65mx krb5kdc[21585](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517697, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.52.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 06:54:57.346753 21600 init.cc:377] Logged in from keytab as kudu/127.19.52.126@KRBTEST.COM (short username kudu)
I20260812 06:54:57.347877 21600 webserver.cc:460] Webserver started at http://127.19.52.126:33905/ using document root <none> and password file <none>
I20260812 06:54:57.348635 21600 fs_manager.cc:359] Metadata directory not provided
I20260812 06:54:57.348742 21600 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:54:57.349063 21600 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 06:54:57.351410 21600 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786517687384214-19665-0/minicluster-data/master-0/data/instance:
uuid: "edbfe34f29a1474d96de7496f4a45b6f"
format_stamp: "Formatted at 2026-08-12 06:54:57 on dist-test-slave-65mx"
server_key: "d164cd9e4f7fae1f5faa76cb6cfceb49"
server_key_iv: "2facfff726e4ac608db9c8e19fe23f7d"
server_key_version: "encryptionkey@0"
I20260812 06:54:57.352064 21600 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786517687384214-19665-0/minicluster-data/master-0/wal/instance:
uuid: "edbfe34f29a1474d96de7496f4a45b6f"
format_stamp: "Formatted at 2026-08-12 06:54:57 on dist-test-slave-65mx"
server_key: "d164cd9e4f7fae1f5faa76cb6cfceb49"
server_key_iv: "2facfff726e4ac608db9c8e19fe23f7d"
server_key_version: "encryptionkey@0"
I20260812 06:54:57.356455 21600 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.007s sys 0.000s
I20260812 06:54:57.359170 21615 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 06:54:57.360486 21600 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260812 06:54:57.360713 21600 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786517687384214-19665-0/minicluster-data/master-0/data,/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786517687384214-19665-0/minicluster-data/master-0/wal
uuid: "edbfe34f29a1474d96de7496f4a45b6f"
format_stamp: "Formatted at 2026-08-12 06:54:57 on dist-test-slave-65mx"
server_key: "d164cd9e4f7fae1f5faa76cb6cfceb49"
server_key_iv: "2facfff726e4ac608db9c8e19fe23f7d"
server_key_version: "encryptionkey@0"
I20260812 06:54:57.360841 21600 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786517687384214-19665-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786517687384214-19665-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786517687384214-19665-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:54:57.380146 21600 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 06:54:57.383584 21600 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 06:54:57.383810 21600 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:54:57.392473 21600 rpc_server.cc:307] RPC server started. Bound to: 127.19.52.126:43141
I20260812 06:54:57.392504 21688 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.52.126:43141 every 8 connection(s)
I20260812 06:54:57.393477 21600 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786517687384214-19665-0/minicluster-data/master-0/data/info.pb
I20260812 06:54:57.396229 21689 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 06:54:57.399107 19665 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu as pid 21600
I20260812 06:54:57.399233 19665 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786517687384214-19665-0/minicluster-data/master-0/wal/instance
I20260812 06:54:57.399513 19665 external_mini_cluster.cc:1428] Setting key fb4ee7b46555843575805ce146d6c163
I20260812 06:54:57.402328 21689 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P edbfe34f29a1474d96de7496f4a45b6f: Bootstrap starting.
I20260812 06:54:57.405053 21689 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P edbfe34f29a1474d96de7496f4a45b6f: Neither blocks nor log segments found. Creating new log.
I20260812 06:54:57.405997 21689 log.cc:824] T 00000000000000000000000000000000 P edbfe34f29a1474d96de7496f4a45b6f: Log is configured to *not* fsync() on all Append() calls
I20260812 06:54:57.408521 21689 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P edbfe34f29a1474d96de7496f4a45b6f: No bootstrap required, opened a new log
Aug 12 06:54:57 dist-test-slave-65mx krb5kdc[21585](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517697, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.19.52.126@KRBTEST.COM
I20260812 06:54:57.413810 21689 raft_consensus.cc:348] T 00000000000000000000000000000000 P edbfe34f29a1474d96de7496f4a45b6f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "edbfe34f29a1474d96de7496f4a45b6f" member_type: VOTER last_known_addr { host: "127.19.52.126" port: 43141 } }
I20260812 06:54:57.414324 21689 raft_consensus.cc:374] T 00000000000000000000000000000000 P edbfe34f29a1474d96de7496f4a45b6f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 06:54:57.414433 21689 raft_consensus.cc:729] T 00000000000000000000000000000000 P edbfe34f29a1474d96de7496f4a45b6f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: edbfe34f29a1474d96de7496f4a45b6f, State: Initialized, Role: FOLLOWER
I20260812 06:54:57.414966 21689 consensus_queue.cc:260] T 00000000000000000000000000000000 P edbfe34f29a1474d96de7496f4a45b6f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "edbfe34f29a1474d96de7496f4a45b6f" member_type: VOTER last_known_addr { host: "127.19.52.126" port: 43141 } }
I20260812 06:54:57.415160 21689 raft_consensus.cc:388] T 00000000000000000000000000000000 P edbfe34f29a1474d96de7496f4a45b6f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 06:54:57.415237 21689 raft_consensus.cc:482] T 00000000000000000000000000000000 P edbfe34f29a1474d96de7496f4a45b6f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 06:54:57.415350 21689 raft_consensus.cc:3037] T 00000000000000000000000000000000 P edbfe34f29a1474d96de7496f4a45b6f [term 0 FOLLOWER]: Advancing to term 1
I20260812 06:54:57.416623 21689 raft_consensus.cc:504] T 00000000000000000000000000000000 P edbfe34f29a1474d96de7496f4a45b6f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "edbfe34f29a1474d96de7496f4a45b6f" member_type: VOTER last_known_addr { host: "127.19.52.126" port: 43141 } }
I20260812 06:54:57.417034 21693 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:54:57.400964 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46606 (local address 127.19.52.126:43141)
0812 06:54:57.401403 (+ 439us) server_negotiation.cc:206] Beginning negotiation
0812 06:54:57.401419 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0812 06:54:57.401475 (+ 56us) server_negotiation.cc:407] Connection header received
0812 06:54:57.402095 (+ 620us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:54:57.402110 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:54:57.402488 (+ 378us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:54:57.402765 (+ 277us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:54:57.403683 (+ 918us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:54:57.404786 (+ 1103us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:54:57.406050 (+ 1264us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:54:57.406741 (+ 691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:54:57.406939 (+ 198us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:54:57.410286 (+ 3347us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:54:57.410292 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:54:57.410310 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:54:57.410344 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:54:57.412665 (+ 2321us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:54:57.413217 (+ 552us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:54:57.413224 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:54:57.413231 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:54:57.413416 (+ 185us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:54:57.413780 (+ 364us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:54:57.413784 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:54:57.413785 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:54:57.414573 (+ 788us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:54:57.414795 (+ 222us) server_negotiation.cc:1035] Waiting for connection context
0812 06:54:57.415624 (+ 829us) server_negotiation.cc:299] Negotiation successful
0812 06:54:57.415938 (+ 314us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":289,"thread_start_us":147,"threads_started":1}
I20260812 06:54:57.417021 21689 leader_election.cc:302] T 00000000000000000000000000000000 P edbfe34f29a1474d96de7496f4a45b6f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: edbfe34f29a1474d96de7496f4a45b6f; no voters:
I20260812 06:54:57.417383 21689 leader_election.cc:288] T 00000000000000000000000000000000 P edbfe34f29a1474d96de7496f4a45b6f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 06:54:57.417619 21696 raft_consensus.cc:2781] T 00000000000000000000000000000000 P edbfe34f29a1474d96de7496f4a45b6f [term 1 FOLLOWER]: Leader election won for term 1
I20260812 06:54:57.417887 21696 raft_consensus.cc:686] T 00000000000000000000000000000000 P edbfe34f29a1474d96de7496f4a45b6f [term 1 LEADER]: Becoming Leader. State: Replica: edbfe34f29a1474d96de7496f4a45b6f, State: Running, Role: LEADER
I20260812 06:54:57.418208 21696 consensus_queue.cc:237] T 00000000000000000000000000000000 P edbfe34f29a1474d96de7496f4a45b6f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "edbfe34f29a1474d96de7496f4a45b6f" member_type: VOTER last_known_addr { host: "127.19.52.126" port: 43141 } }
I20260812 06:54:57.418776 21689 sys_catalog.cc:564] T 00000000000000000000000000000000 P edbfe34f29a1474d96de7496f4a45b6f [sys.catalog]: configured and running, proceeding with master startup.
I20260812 06:54:57.419427 21696 sys_catalog.cc:455] T 00000000000000000000000000000000 P edbfe34f29a1474d96de7496f4a45b6f [sys.catalog]: SysCatalogTable state changed. Reason: New leader edbfe34f29a1474d96de7496f4a45b6f. Latest consensus state: current_term: 1 leader_uuid: "edbfe34f29a1474d96de7496f4a45b6f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "edbfe34f29a1474d96de7496f4a45b6f" member_type: VOTER last_known_addr { host: "127.19.52.126" port: 43141 } } }
I20260812 06:54:57.419557 21696 sys_catalog.cc:458] T 00000000000000000000000000000000 P edbfe34f29a1474d96de7496f4a45b6f [sys.catalog]: This master's current role is: LEADER
I20260812 06:54:57.419855 21698 sys_catalog.cc:455] T 00000000000000000000000000000000 P edbfe34f29a1474d96de7496f4a45b6f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "edbfe34f29a1474d96de7496f4a45b6f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "edbfe34f29a1474d96de7496f4a45b6f" member_type: VOTER last_known_addr { host: "127.19.52.126" port: 43141 } } }
I20260812 06:54:57.419961 21698 sys_catalog.cc:458] T 00000000000000000000000000000000 P edbfe34f29a1474d96de7496f4a45b6f [sys.catalog]: This master's current role is: LEADER
I20260812 06:54:57.421072 21702 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 06:54:57.424111 21702 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 06:54:57.429719 21702 catalog_manager.cc:1347] Generated new cluster ID: 5346598bda5c4e0aa5192ceba63a439f
I20260812 06:54:57.429806 21702 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 06:54:57.452945 21702 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 06:54:57.454371 21702 catalog_manager.cc:1504] Loading token signing keys...
I20260812 06:54:57.460421 21702 catalog_manager.cc:5868] T 00000000000000000000000000000000 P edbfe34f29a1474d96de7496f4a45b6f: Generated new TSK 0
I20260812 06:54:57.461304 21702 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.19.52.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.52.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.52.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786517687384214-19665-0/krb5kdc/kudu_127.19.52.65.keytab.
Entry for principal kudu/127.19.52.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786517687384214-19665-0/krb5kdc/kudu_127.19.52.65.keytab.
I20260812 06:54:57.504446 19665 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu
/tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786517687384214-19665-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786517687384214-19665-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786517687384214-19665-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786517687384214-19665-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.52.65:0
--local_ip_for_outbound_sockets=127.19.52.65
--webserver_interface=127.19.52.65
--webserver_port=0
--tserver_master_addrs=127.19.52.126:43141
--keytab_file=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786517687384214-19665-0/krb5kdc/kudu_127.19.52.65.keytab
--principal=kudu/127.19.52.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.52.84:42943
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786517687384214-19665-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786517687384214-19665-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786517687384214-19665-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 06:54:57.621078 21719 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:54:57.621443 21719 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:54:57.621549 21719 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 06:54:57.625769 21719 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 06:54:57.625890 21719 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 06:54:57.625972 21719 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260812 06:54:57.626019 21719 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260812 06:54:57.626077 21719 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.52.65
I20260812 06:54:57.631641 21719 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.52.84:42943
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786517687384214-19665-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786517687384214-19665-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786517687384214-19665-0/krb5kdc/kudu_127.19.52.65.keytab
--principal=kudu/127.19.52.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.52.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786517687384214-19665-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.19.52.65
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.19.52.126:43141
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21719
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.52.65
--log_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786517687384214-19665-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 06:15:54 UTC on ccycloud.quasar-rnfzce.root.comops.site
build id 4581989
I20260812 06:54:57.632903 21719 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:54:57.634011 21719 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 06:54:57.644467 21726 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:54:57.644467 21730 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:54:57.644471 21725 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:54:57.644701 21719 server_base.cc:1034] running on GCE node
I20260812 06:54:57.645331 21719 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:54:57.646016 21719 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:54:57.647207 21719 hybrid_clock.cc:648] HybridClock initialized: now 1786517697647192 us; error 29 us; skew 500 ppm
Aug 12 06:54:57 dist-test-slave-65mx krb5kdc[21585](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517697, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.52.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 06:54:57.656790 21719 init.cc:377] Logged in from keytab as kudu/127.19.52.65@KRBTEST.COM (short username kudu)
I20260812 06:54:57.657924 21719 webserver.cc:460] Webserver started at http://127.19.52.65:33495/ using document root <none> and password file <none>
I20260812 06:54:57.658779 21719 fs_manager.cc:359] Metadata directory not provided
I20260812 06:54:57.658871 21719 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:54:57.659165 21719 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 06:54:57.661568 21719 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786517687384214-19665-0/minicluster-data/ts-0/data/instance:
uuid: "cf0e18c976a24d659861decac8e6b157"
format_stamp: "Formatted at 2026-08-12 06:54:57 on dist-test-slave-65mx"
server_key: "771723e8086640a42946471c6947d7ac"
server_key_iv: "69d7913ea2770b7fab17158427627227"
server_key_version: "encryptionkey@0"
I20260812 06:54:57.662243 21719 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786517687384214-19665-0/minicluster-data/ts-0/wal/instance:
uuid: "cf0e18c976a24d659861decac8e6b157"
format_stamp: "Formatted at 2026-08-12 06:54:57 on dist-test-slave-65mx"
server_key: "771723e8086640a42946471c6947d7ac"
server_key_iv: "69d7913ea2770b7fab17158427627227"
server_key_version: "encryptionkey@0"
I20260812 06:54:57.666769 21719 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.004s
I20260812 06:54:57.669653 21737 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 06:54:57.671052 21719 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 06:54:57.671195 21719 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786517687384214-19665-0/minicluster-data/ts-0/data,/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786517687384214-19665-0/minicluster-data/ts-0/wal
uuid: "cf0e18c976a24d659861decac8e6b157"
format_stamp: "Formatted at 2026-08-12 06:54:57 on dist-test-slave-65mx"
server_key: "771723e8086640a42946471c6947d7ac"
server_key_iv: "69d7913ea2770b7fab17158427627227"
server_key_version: "encryptionkey@0"
I20260812 06:54:57.671322 21719 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786517687384214-19665-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786517687384214-19665-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786517687384214-19665-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:54:57.703804 21719 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 06:54:57.707178 21719 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 06:54:57.707407 21719 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:54:57.709088 21719 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 06:54:57.709152 21719 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 06:54:57.709218 21719 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 06:54:57.709249 21719 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 12 06:54:57 dist-test-slave-65mx krb5kdc[21585](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517697, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.52.65@KRBTEST.COM for kudu/127.19.52.126@KRBTEST.COM
I20260812 06:54:57.721154 21719 rpc_server.cc:307] RPC server started. Bound to: 127.19.52.65:45981
I20260812 06:54:57.721276 21868 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.52.65:45981 every 8 connection(s)
I20260812 06:54:57.722136 21719 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786517687384214-19665-0/minicluster-data/ts-0/data/info.pb
I20260812 06:54:57.722908 19665 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu as pid 21719
I20260812 06:54:57.723044 19665 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786517687384214-19665-0/minicluster-data/ts-0/wal/instance
I20260812 06:54:57.723369 19665 external_mini_cluster.cc:1428] Setting key 5d3d09c2224c6a8e036c6d36436dfd86
I20260812 06:54:57.725791 21693 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:54:57.709916 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.52.65:55403 (local address 127.19.52.126:43141)
0812 06:54:57.710050 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0812 06:54:57.710054 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 06:54:57.711292 (+ 1238us) server_negotiation.cc:407] Connection header received
0812 06:54:57.712716 (+ 1424us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:54:57.712722 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:54:57.712795 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:54:57.712849 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:54:57.714556 (+ 1707us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:54:57.715525 (+ 969us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:54:57.717277 (+ 1752us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:54:57.717665 (+ 388us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:54:57.717785 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:54:57.721312 (+ 3527us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:54:57.721318 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:54:57.721321 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:54:57.721360 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:54:57.723329 (+ 1969us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:54:57.723960 (+ 631us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:54:57.723964 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:54:57.723966 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:54:57.724090 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:54:57.724512 (+ 422us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:54:57.724515 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:54:57.724516 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:54:57.724827 (+ 311us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:54:57.724945 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0812 06:54:57.725473 (+ 528us) server_negotiation.cc:299] Negotiation successful
0812 06:54:57.725608 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260812 06:54:57.726608 21746 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:54:57.710515 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.126:43141 (local address 127.19.52.65:55403)
0812 06:54:57.711097 (+ 582us) negotiation.cc:107] Waiting for socket to connect
0812 06:54:57.711155 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0812 06:54:57.712300 (+ 1145us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:54:57.713246 (+ 946us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:54:57.713258 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:54:57.713657 (+ 399us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:54:57.714391 (+ 734us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:54:57.714407 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:54:57.715681 (+ 1274us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:54:57.715686 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:54:57.717155 (+ 1469us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:54:57.717166 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:54:57.717846 (+ 680us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:54:57.717849 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:54:57.718007 (+ 158us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:54:57.718737 (+ 730us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 06:54:57.718761 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:54:57.721072 (+ 2311us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 06:54:57.723465 (+ 2393us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:54:57.723473 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:54:57.723488 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:54:57.723851 (+ 363us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:54:57.724207 (+ 356us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:54:57.724210 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:54:57.724212 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:54:57.724389 (+ 177us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:54:57.724946 (+ 557us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 06:54:57.724954 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 06:54:57.725222 (+ 268us) client_negotiation.cc:769] Sending connection context
0812 06:54:57.725415 (+ 193us) client_negotiation.cc:240] Negotiation successful
0812 06:54:57.725758 (+ 343us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":355,"thread_start_us":151,"threads_started":1}
I20260812 06:54:57.727967 21869 heartbeater.cc:347] Connected to a master server at 127.19.52.126:43141
I20260812 06:54:57.728123 21869 heartbeater.cc:464] Registering TS with master...
I20260812 06:54:57.728734 21869 heartbeater.cc:511] Master 127.19.52.126:43141 requested a full tablet report, sending...
I20260812 06:54:57.730624 21633 ts_manager.cc:194] Registered new tserver with Master: cf0e18c976a24d659861decac8e6b157 (127.19.52.65:45981)
I20260812 06:54:57.732322 21633 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.52.65@KRBTEST.COM'} at 127.19.52.65:55403
WARNING: no policy specified for kudu/127.19.52.66@KRBTEST.COM; defaulting to no policy
I20260812 06:54:57.740505 21693 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:54:57.730738 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.52.65:36797 (local address 127.19.52.126:43141)
0812 06:54:57.730881 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0812 06:54:57.730888 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 06:54:57.731253 (+ 365us) server_negotiation.cc:407] Connection header received
0812 06:54:57.731397 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:54:57.731400 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:54:57.731452 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:54:57.731554 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:54:57.732233 (+ 679us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:54:57.733272 (+ 1039us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:54:57.734624 (+ 1352us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:54:57.734935 (+ 311us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:54:57.735052 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:54:57.736604 (+ 1552us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:54:57.736612 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:54:57.736617 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:54:57.736680 (+ 63us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:54:57.738567 (+ 1887us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:54:57.739085 (+ 518us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:54:57.739088 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:54:57.739090 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:54:57.739196 (+ 106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:54:57.739658 (+ 462us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:54:57.739661 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:54:57.739662 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:54:57.739878 (+ 216us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:54:57.739967 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0812 06:54:57.740273 (+ 306us) server_negotiation.cc:299] Negotiation successful
0812 06:54:57.740380 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":51}
I20260812 06:54:57.740513 21877 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:54:57.730645 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.126:43141 (local address 127.19.52.65:36797)
0812 06:54:57.731086 (+ 441us) negotiation.cc:107] Waiting for socket to connect
0812 06:54:57.731105 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0812 06:54:57.731283 (+ 178us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:54:57.731686 (+ 403us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:54:57.731690 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:54:57.731948 (+ 258us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:54:57.732085 (+ 137us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:54:57.732092 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:54:57.733432 (+ 1340us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:54:57.733439 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:54:57.734490 (+ 1051us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:54:57.734502 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:54:57.735099 (+ 597us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:54:57.735103 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:54:57.735188 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:54:57.735779 (+ 591us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 06:54:57.735790 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:54:57.736266 (+ 476us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 06:54:57.738719 (+ 2453us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:54:57.738723 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:54:57.738725 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:54:57.738979 (+ 254us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:54:57.739349 (+ 370us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:54:57.739352 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:54:57.739353 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:54:57.739461 (+ 108us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:54:57.740012 (+ 551us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 06:54:57.740015 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 06:54:57.740113 (+ 98us) client_negotiation.cc:769] Sending connection context
0812 06:54:57.740204 (+ 91us) client_negotiation.cc:240] Negotiation successful
0812 06:54:57.740337 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":323,"thread_start_us":122,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.52.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.52.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786517687384214-19665-0/krb5kdc/kudu_127.19.52.66.keytab.
Entry for principal kudu/127.19.52.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786517687384214-19665-0/krb5kdc/kudu_127.19.52.66.keytab.
I20260812 06:54:57.760087 19665 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu
/tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786517687384214-19665-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786517687384214-19665-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786517687384214-19665-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786517687384214-19665-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.52.66:0
--local_ip_for_outbound_sockets=127.19.52.66
--webserver_interface=127.19.52.66
--webserver_port=0
--tserver_master_addrs=127.19.52.126:43141
--keytab_file=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786517687384214-19665-0/krb5kdc/kudu_127.19.52.66.keytab
--principal=kudu/127.19.52.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.52.84:42943
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786517687384214-19665-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786517687384214-19665-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786517687384214-19665-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 06:54:57.878799 21879 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:54:57.879103 21879 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:54:57.879154 21879 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 06:54:57.883143 21879 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 06:54:57.883205 21879 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 06:54:57.883246 21879 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260812 06:54:57.883261 21879 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260812 06:54:57.883292 21879 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.52.66
I20260812 06:54:57.888787 21879 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.52.84:42943
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786517687384214-19665-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786517687384214-19665-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786517687384214-19665-0/krb5kdc/kudu_127.19.52.66.keytab
--principal=kudu/127.19.52.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.52.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786517687384214-19665-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.19.52.66
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.19.52.126:43141
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21879
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.52.66
--log_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786517687384214-19665-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 06:15:54 UTC on ccycloud.quasar-rnfzce.root.comops.site
build id 4581989
I20260812 06:54:57.889887 21879 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:54:57.890878 21879 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 06:54:57.900981 21887 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:54:57.900986 21890 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:54:57.900990 21888 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:54:57.901211 21879 server_base.cc:1034] running on GCE node
I20260812 06:54:57.901793 21879 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:54:57.902418 21879 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:54:57.903594 21879 hybrid_clock.cc:648] HybridClock initialized: now 1786517697903579 us; error 28 us; skew 500 ppm
Aug 12 06:54:57 dist-test-slave-65mx krb5kdc[21585](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517697, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.52.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 06:54:57.906803 21879 init.cc:377] Logged in from keytab as kudu/127.19.52.66@KRBTEST.COM (short username kudu)
I20260812 06:54:57.908113 21879 webserver.cc:460] Webserver started at http://127.19.52.66:36627/ using document root <none> and password file <none>
I20260812 06:54:57.908821 21879 fs_manager.cc:359] Metadata directory not provided
I20260812 06:54:57.908924 21879 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:54:57.909188 21879 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 06:54:57.911379 21879 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786517687384214-19665-0/minicluster-data/ts-1/data/instance:
uuid: "adadc55f535f4b0dbe3d04e41251d456"
format_stamp: "Formatted at 2026-08-12 06:54:57 on dist-test-slave-65mx"
server_key: "e59820414636e6e321f043f1b8c74cb1"
server_key_iv: "2f8a46dba4566870d47d2793fcbf89a8"
server_key_version: "encryptionkey@0"
I20260812 06:54:57.911983 21879 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786517687384214-19665-0/minicluster-data/ts-1/wal/instance:
uuid: "adadc55f535f4b0dbe3d04e41251d456"
format_stamp: "Formatted at 2026-08-12 06:54:57 on dist-test-slave-65mx"
server_key: "e59820414636e6e321f043f1b8c74cb1"
server_key_iv: "2f8a46dba4566870d47d2793fcbf89a8"
server_key_version: "encryptionkey@0"
I20260812 06:54:57.916054 21879 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260812 06:54:57.918884 21897 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 06:54:57.920454 21879 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260812 06:54:57.920648 21879 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786517687384214-19665-0/minicluster-data/ts-1/data,/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786517687384214-19665-0/minicluster-data/ts-1/wal
uuid: "adadc55f535f4b0dbe3d04e41251d456"
format_stamp: "Formatted at 2026-08-12 06:54:57 on dist-test-slave-65mx"
server_key: "e59820414636e6e321f043f1b8c74cb1"
server_key_iv: "2f8a46dba4566870d47d2793fcbf89a8"
server_key_version: "encryptionkey@0"
I20260812 06:54:57.920786 21879 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786517687384214-19665-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786517687384214-19665-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786517687384214-19665-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:54:57.938654 21879 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 06:54:57.942209 21879 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 06:54:57.942430 21879 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:54:57.943976 21879 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 06:54:57.944053 21879 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 06:54:57.944128 21879 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 06:54:57.944180 21879 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 12 06:54:57 dist-test-slave-65mx krb5kdc[21585](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517697, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.52.66@KRBTEST.COM for kudu/127.19.52.126@KRBTEST.COM
I20260812 06:54:57.955561 21879 rpc_server.cc:307] RPC server started. Bound to: 127.19.52.66:36983
I20260812 06:54:57.955760 22033 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.52.66:36983 every 8 connection(s)
I20260812 06:54:57.956782 21879 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786517687384214-19665-0/minicluster-data/ts-1/data/info.pb
I20260812 06:54:57.958122 19665 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu as pid 21879
I20260812 06:54:57.958303 19665 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786517687384214-19665-0/minicluster-data/ts-1/wal/instance
I20260812 06:54:57.958637 19665 external_mini_cluster.cc:1428] Setting key cfb20a6b6c1cccc90bda69db92ed669b
I20260812 06:54:57.959225 21693 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:54:57.944907 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.52.66:48689 (local address 127.19.52.126:43141)
0812 06:54:57.945057 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0812 06:54:57.945061 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 06:54:57.946234 (+ 1173us) server_negotiation.cc:407] Connection header received
0812 06:54:57.947363 (+ 1129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:54:57.947370 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:54:57.947449 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:54:57.947566 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:54:57.949139 (+ 1573us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:54:57.949689 (+ 550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:54:57.950992 (+ 1303us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:54:57.951233 (+ 241us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:54:57.951357 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:54:57.954755 (+ 3398us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:54:57.954759 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:54:57.954761 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:54:57.954786 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:54:57.956528 (+ 1742us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:54:57.957302 (+ 774us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:54:57.957308 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:54:57.957314 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:54:57.957421 (+ 107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:54:57.957869 (+ 448us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:54:57.957874 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:54:57.957876 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:54:57.958092 (+ 216us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:54:57.958212 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0812 06:54:57.958878 (+ 666us) server_negotiation.cc:299] Negotiation successful
0812 06:54:57.959030 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260812 06:54:57.960240 21906 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:54:57.945465 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.126:43141 (local address 127.19.52.66:48689)
0812 06:54:57.946050 (+ 585us) negotiation.cc:107] Waiting for socket to connect
0812 06:54:57.946098 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0812 06:54:57.947076 (+ 978us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:54:57.947776 (+ 700us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:54:57.947786 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:54:57.948303 (+ 517us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:54:57.948975 (+ 672us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:54:57.948992 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:54:57.949831 (+ 839us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:54:57.949835 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:54:57.950845 (+ 1010us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:54:57.950856 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:54:57.951417 (+ 561us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:54:57.951422 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:54:57.951630 (+ 208us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:54:57.952284 (+ 654us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 06:54:57.952301 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:54:57.954566 (+ 2265us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 06:54:57.956673 (+ 2107us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:54:57.956678 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:54:57.956690 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:54:57.957165 (+ 475us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:54:57.957566 (+ 401us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:54:57.957571 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:54:57.957574 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:54:57.957754 (+ 180us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:54:57.958227 (+ 473us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 06:54:57.958236 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 06:54:57.958601 (+ 365us) client_negotiation.cc:769] Sending connection context
0812 06:54:57.958829 (+ 228us) client_negotiation.cc:240] Negotiation successful
0812 06:54:57.959175 (+ 346us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":339,"thread_start_us":148,"threads_started":1}
I20260812 06:54:57.961565 22035 heartbeater.cc:347] Connected to a master server at 127.19.52.126:43141
I20260812 06:54:57.961709 22035 heartbeater.cc:464] Registering TS with master...
I20260812 06:54:57.962270 22035 heartbeater.cc:511] Master 127.19.52.126:43141 requested a full tablet report, sending...
I20260812 06:54:57.963479 21634 ts_manager.cc:194] Registered new tserver with Master: adadc55f535f4b0dbe3d04e41251d456 (127.19.52.66:36983)
I20260812 06:54:57.964203 21634 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.52.66@KRBTEST.COM'} at 127.19.52.66:48689
WARNING: no policy specified for kudu/127.19.52.67@KRBTEST.COM; defaulting to no policy
I20260812 06:54:57.972877 22042 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:54:57.964189 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.126:43141 (local address 127.19.52.66:59187)
0812 06:54:57.964512 (+ 323us) negotiation.cc:107] Waiting for socket to connect
0812 06:54:57.964530 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0812 06:54:57.964659 (+ 129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:54:57.964954 (+ 295us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:54:57.964960 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:54:57.965220 (+ 260us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:54:57.965390 (+ 170us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:54:57.965400 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:54:57.966608 (+ 1208us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:54:57.966612 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:54:57.967668 (+ 1056us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:54:57.967682 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:54:57.968244 (+ 562us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:54:57.968248 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:54:57.968356 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:54:57.968955 (+ 599us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 06:54:57.968972 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:54:57.969448 (+ 476us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 06:54:57.971361 (+ 1913us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:54:57.971364 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:54:57.971365 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:54:57.971556 (+ 191us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:54:57.971869 (+ 313us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:54:57.971871 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:54:57.971873 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:54:57.971951 (+ 78us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:54:57.972350 (+ 399us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 06:54:57.972352 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 06:54:57.972426 (+ 74us) client_negotiation.cc:769] Sending connection context
0812 06:54:57.972506 (+ 80us) client_negotiation.cc:240] Negotiation successful
0812 06:54:57.972700 (+ 194us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":211,"thread_start_us":122,"threads_started":1}
I20260812 06:54:57.972903 21693 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:54:57.964584 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.52.66:59187 (local address 127.19.52.126:43141)
0812 06:54:57.964714 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0812 06:54:57.964720 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 06:54:57.964734 (+ 14us) server_negotiation.cc:407] Connection header received
0812 06:54:57.964773 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:54:57.964776 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:54:57.964823 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:54:57.964914 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:54:57.965538 (+ 624us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:54:57.966426 (+ 888us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:54:57.967810 (+ 1384us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:54:57.968109 (+ 299us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:54:57.968202 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:54:57.969604 (+ 1402us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:54:57.969607 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:54:57.969609 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:54:57.969633 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:54:57.971254 (+ 1621us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:54:57.971647 (+ 393us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:54:57.971650 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:54:57.971651 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:54:57.971766 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:54:57.972040 (+ 274us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:54:57.972042 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:54:57.972044 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:54:57.972264 (+ 220us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:54:57.972349 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0812 06:54:57.972636 (+ 287us) server_negotiation.cc:299] Negotiation successful
0812 06:54:57.972738 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.52.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.52.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786517687384214-19665-0/krb5kdc/kudu_127.19.52.67.keytab.
Entry for principal kudu/127.19.52.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786517687384214-19665-0/krb5kdc/kudu_127.19.52.67.keytab.
I20260812 06:54:57.991885 19665 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu
/tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786517687384214-19665-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786517687384214-19665-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786517687384214-19665-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786517687384214-19665-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.52.67:0
--local_ip_for_outbound_sockets=127.19.52.67
--webserver_interface=127.19.52.67
--webserver_port=0
--tserver_master_addrs=127.19.52.126:43141
--keytab_file=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786517687384214-19665-0/krb5kdc/kudu_127.19.52.67.keytab
--principal=kudu/127.19.52.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.52.84:42943
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786517687384214-19665-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786517687384214-19665-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786517687384214-19665-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 06:54:58.106268 22044 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:54:58.106534 22044 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:54:58.106613 22044 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 06:54:58.110625 22044 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 06:54:58.110684 22044 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 06:54:58.110724 22044 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260812 06:54:58.110738 22044 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260812 06:54:58.110827 22044 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.52.67
I20260812 06:54:58.116061 22044 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.52.84:42943
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786517687384214-19665-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786517687384214-19665-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786517687384214-19665-0/krb5kdc/kudu_127.19.52.67.keytab
--principal=kudu/127.19.52.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.52.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786517687384214-19665-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.19.52.67
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.19.52.126:43141
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22044
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.52.67
--log_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786517687384214-19665-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 06:15:54 UTC on ccycloud.quasar-rnfzce.root.comops.site
build id 4581989
I20260812 06:54:58.117182 22044 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:54:58.118254 22044 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 06:54:58.128907 22052 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:54:58.128908 22055 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:54:58.128907 22050 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:54:58.129330 22044 server_base.cc:1034] running on GCE node
I20260812 06:54:58.130141 22044 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:54:58.130872 22044 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:54:58.132090 22044 hybrid_clock.cc:648] HybridClock initialized: now 1786517698132068 us; error 34 us; skew 500 ppm
Aug 12 06:54:58 dist-test-slave-65mx krb5kdc[21585](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517698, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.52.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 06:54:58.135258 22044 init.cc:377] Logged in from keytab as kudu/127.19.52.67@KRBTEST.COM (short username kudu)
I20260812 06:54:58.136291 22044 webserver.cc:460] Webserver started at http://127.19.52.67:43583/ using document root <none> and password file <none>
I20260812 06:54:58.137156 22044 fs_manager.cc:359] Metadata directory not provided
I20260812 06:54:58.137259 22044 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:54:58.137508 22044 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 06:54:58.139672 22044 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786517687384214-19665-0/minicluster-data/ts-2/data/instance:
uuid: "6a49c31e22d740fab75a5123639a3502"
format_stamp: "Formatted at 2026-08-12 06:54:58 on dist-test-slave-65mx"
server_key: "d3af5467541ae72bd4409a37da16f124"
server_key_iv: "89077f00e7dc061813d84177376fd195"
server_key_version: "encryptionkey@0"
I20260812 06:54:58.140259 22044 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786517687384214-19665-0/minicluster-data/ts-2/wal/instance:
uuid: "6a49c31e22d740fab75a5123639a3502"
format_stamp: "Formatted at 2026-08-12 06:54:58 on dist-test-slave-65mx"
server_key: "d3af5467541ae72bd4409a37da16f124"
server_key_iv: "89077f00e7dc061813d84177376fd195"
server_key_version: "encryptionkey@0"
I20260812 06:54:58.144317 22044 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.007s sys 0.000s
I20260812 06:54:58.147147 22062 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 06:54:58.148684 22044 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 06:54:58.148801 22044 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786517687384214-19665-0/minicluster-data/ts-2/data,/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786517687384214-19665-0/minicluster-data/ts-2/wal
uuid: "6a49c31e22d740fab75a5123639a3502"
format_stamp: "Formatted at 2026-08-12 06:54:58 on dist-test-slave-65mx"
server_key: "d3af5467541ae72bd4409a37da16f124"
server_key_iv: "89077f00e7dc061813d84177376fd195"
server_key_version: "encryptionkey@0"
I20260812 06:54:58.148908 22044 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786517687384214-19665-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786517687384214-19665-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786517687384214-19665-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:54:58.160843 22044 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 06:54:58.164188 22044 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 06:54:58.164350 22044 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:54:58.166078 22044 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 06:54:58.166139 22044 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 06:54:58.166205 22044 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 06:54:58.166239 22044 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 12 06:54:58 dist-test-slave-65mx krb5kdc[21585](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517698, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.52.67@KRBTEST.COM for kudu/127.19.52.126@KRBTEST.COM
I20260812 06:54:58.178983 22044 rpc_server.cc:307] RPC server started. Bound to: 127.19.52.67:44177
I20260812 06:54:58.179054 22191 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.52.67:44177 every 8 connection(s)
I20260812 06:54:58.180449 22044 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786517687384214-19665-0/minicluster-data/ts-2/data/info.pb
I20260812 06:54:58.182937 21693 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:54:58.167149 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.52.67:33561 (local address 127.19.52.126:43141)
0812 06:54:58.167301 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0812 06:54:58.167305 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 06:54:58.168402 (+ 1097us) server_negotiation.cc:407] Connection header received
0812 06:54:58.169689 (+ 1287us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:54:58.169693 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:54:58.169773 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:54:58.169878 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:54:58.171252 (+ 1374us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:54:58.171932 (+ 680us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:54:58.173299 (+ 1367us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:54:58.173529 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:54:58.173670 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:54:58.178165 (+ 4495us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:54:58.178169 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:54:58.178171 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:54:58.178200 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:54:58.180222 (+ 2022us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:54:58.181103 (+ 881us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:54:58.181108 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:54:58.181110 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:54:58.181220 (+ 110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:54:58.181674 (+ 454us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:54:58.181677 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:54:58.181679 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:54:58.181891 (+ 212us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:54:58.181978 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0812 06:54:58.182690 (+ 712us) server_negotiation.cc:299] Negotiation successful
0812 06:54:58.182812 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260812 06:54:58.183817 22071 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:54:58.167716 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.126:43141 (local address 127.19.52.67:33561)
0812 06:54:58.168234 (+ 518us) negotiation.cc:107] Waiting for socket to connect
0812 06:54:58.168279 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0812 06:54:58.169415 (+ 1136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:54:58.170067 (+ 652us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:54:58.170075 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:54:58.170433 (+ 358us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:54:58.171095 (+ 662us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:54:58.171112 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:54:58.172103 (+ 991us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:54:58.172134 (+ 31us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:54:58.173161 (+ 1027us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:54:58.173172 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:54:58.173731 (+ 559us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:54:58.173736 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:54:58.173940 (+ 204us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:54:58.174926 (+ 986us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 06:54:58.174972 (+ 46us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:54:58.177930 (+ 2958us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 06:54:58.180394 (+ 2464us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:54:58.180401 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:54:58.180425 (+ 24us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:54:58.180899 (+ 474us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:54:58.181389 (+ 490us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:54:58.181392 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:54:58.181394 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:54:58.181570 (+ 176us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:54:58.182162 (+ 592us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 06:54:58.182167 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 06:54:58.182431 (+ 264us) client_negotiation.cc:769] Sending connection context
0812 06:54:58.182604 (+ 173us) client_negotiation.cc:240] Negotiation successful
0812 06:54:58.182880 (+ 276us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":328,"thread_start_us":148,"threads_started":1}
I20260812 06:54:58.185302 22193 heartbeater.cc:347] Connected to a master server at 127.19.52.126:43141
I20260812 06:54:58.185459 22193 heartbeater.cc:464] Registering TS with master...
I20260812 06:54:58.186064 22193 heartbeater.cc:511] Master 127.19.52.126:43141 requested a full tablet report, sending...
I20260812 06:54:58.186323 19665 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu as pid 22044
I20260812 06:54:58.186517 19665 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786517687384214-19665-0/minicluster-data/ts-2/wal/instance
I20260812 06:54:58.186929 19665 external_mini_cluster.cc:1428] Setting key f9857e4d7e30cd01fe6ab01df03cdb0e
I20260812 06:54:58.187386 21633 ts_manager.cc:194] Registered new tserver with Master: 6a49c31e22d740fab75a5123639a3502 (127.19.52.67:44177)
I20260812 06:54:58.188243 21633 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.52.67@KRBTEST.COM'} at 127.19.52.67:33561
I20260812 06:54:58.189370 19665 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 06:54:58.196208 21693 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:54:58.187977 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.52.67:52149 (local address 127.19.52.126:43141)
0812 06:54:58.188097 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0812 06:54:58.188101 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 06:54:58.188240 (+ 139us) server_negotiation.cc:407] Connection header received
0812 06:54:58.188411 (+ 171us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:54:58.188415 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:54:58.188470 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:54:58.188592 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:54:58.189257 (+ 665us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:54:58.189860 (+ 603us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:54:58.190737 (+ 877us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:54:58.190959 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:54:58.191063 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:54:58.192432 (+ 1369us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:54:58.192439 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:54:58.192445 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:54:58.192480 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:54:58.194417 (+ 1937us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:54:58.194887 (+ 470us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:54:58.194892 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:54:58.194894 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:54:58.194982 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:54:58.195289 (+ 307us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:54:58.195295 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:54:58.195297 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:54:58.195490 (+ 193us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:54:58.195609 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0812 06:54:58.195903 (+ 294us) server_negotiation.cc:299] Negotiation successful
0812 06:54:58.196025 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260812 06:54:58.196223 22200 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:54:58.187870 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.126:43141 (local address 127.19.52.67:52149)
0812 06:54:58.188145 (+ 275us) negotiation.cc:107] Waiting for socket to connect
0812 06:54:58.188164 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0812 06:54:58.188282 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:54:58.188726 (+ 444us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:54:58.188729 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:54:58.189004 (+ 275us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:54:58.189138 (+ 134us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:54:58.189146 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:54:58.189998 (+ 852us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:54:58.190002 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:54:58.190604 (+ 602us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:54:58.190613 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:54:58.191078 (+ 465us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:54:58.191082 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:54:58.191186 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:54:58.191885 (+ 699us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 06:54:58.191898 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:54:58.192257 (+ 359us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 06:54:58.194538 (+ 2281us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:54:58.194544 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:54:58.194546 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:54:58.194775 (+ 229us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:54:58.195085 (+ 310us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:54:58.195091 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:54:58.195096 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:54:58.195190 (+ 94us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:54:58.195603 (+ 413us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 06:54:58.195607 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 06:54:58.195719 (+ 112us) client_negotiation.cc:769] Sending connection context
0812 06:54:58.195836 (+ 117us) client_negotiation.cc:240] Negotiation successful
0812 06:54:58.196014 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":191,"mutex_wait_us":3,"thread_start_us":107,"threads_started":1}
Aug 12 06:54:58 dist-test-slave-65mx krb5kdc[21585](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517698, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 12 06:54:58 dist-test-slave-65mx krb5kdc[21585](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517698, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.19.52.126@KRBTEST.COM
I20260812 06:54:58.220521 21693 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:54:58.211425 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46614 (local address 127.19.52.126:43141)
0812 06:54:58.211563 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0812 06:54:58.211567 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 06:54:58.211580 (+ 13us) server_negotiation.cc:407] Connection header received
0812 06:54:58.211621 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:54:58.211623 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:54:58.211669 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:54:58.211768 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:54:58.212523 (+ 755us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:54:58.213136 (+ 613us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:54:58.214003 (+ 867us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:54:58.214230 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:54:58.214339 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:54:58.217049 (+ 2710us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:54:58.217053 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:54:58.217055 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:54:58.217083 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:54:58.218607 (+ 1524us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:54:58.219077 (+ 470us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:54:58.219081 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:54:58.219083 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:54:58.219176 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:54:58.219485 (+ 309us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:54:58.219490 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:54:58.219492 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:54:58.219780 (+ 288us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:54:58.219932 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0812 06:54:58.220186 (+ 254us) server_negotiation.cc:299] Negotiation successful
0812 06:54:58.220344 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260812 06:54:58.223603 21633 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46614:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 06:54:58.227072 21633 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 06:54:58.242962 22211 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:54:58.236467 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.67:44177 (local address 127.0.0.1:43562)
0812 06:54:58.237040 (+ 573us) negotiation.cc:107] Waiting for socket to connect
0812 06:54:58.237100 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0812 06:54:58.237240 (+ 140us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:54:58.238481 (+ 1241us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:54:58.238483 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:54:58.238495 (+ 12us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 06:54:58.238689 (+ 194us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:54:58.238698 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:54:58.239837 (+ 1139us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:54:58.239844 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:54:58.241121 (+ 1277us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:54:58.241132 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:54:58.242328 (+ 1196us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:54:58.242332 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:54:58.242470 (+ 138us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:54:58.242520 (+ 50us) client_negotiation.cc:769] Sending connection context
0812 06:54:58.242642 (+ 122us) client_negotiation.cc:240] Negotiation successful
0812 06:54:58.242778 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":343,"mutex_wait_us":5,"thread_start_us":146,"threads_started":1}
I20260812 06:54:58.243010 22213 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:54:58.236901 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.65:45981 (local address 127.0.0.1:47220)
0812 06:54:58.237213 (+ 312us) negotiation.cc:107] Waiting for socket to connect
0812 06:54:58.237230 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0812 06:54:58.237324 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:54:58.238437 (+ 1113us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:54:58.238446 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:54:58.238474 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 06:54:58.238729 (+ 255us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:54:58.238740 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:54:58.240339 (+ 1599us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:54:58.240343 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:54:58.241681 (+ 1338us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:54:58.241689 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:54:58.242614 (+ 925us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:54:58.242620 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:54:58.242716 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:54:58.242738 (+ 22us) client_negotiation.cc:769] Sending connection context
0812 06:54:58.242816 (+ 78us) client_negotiation.cc:240] Negotiation successful
0812 06:54:58.242877 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":203,"mutex_wait_us":249,"spinlock_wait_cycles":10752,"thread_start_us":116,"threads_started":1}
I20260812 06:54:58.243132 22215 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:54:58.237399 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47220 (local address 127.19.52.65:45981)
0812 06:54:58.237894 (+ 495us) server_negotiation.cc:206] Beginning negotiation
0812 06:54:58.237903 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 06:54:58.237931 (+ 28us) server_negotiation.cc:407] Connection header received
0812 06:54:58.238014 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:54:58.238023 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:54:58.238259 (+ 236us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:54:58.238429 (+ 170us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 06:54:58.238885 (+ 456us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:54:58.240192 (+ 1307us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:54:58.241807 (+ 1615us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:54:58.242447 (+ 640us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:54:58.242552 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:54:58.242694 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0812 06:54:58.242864 (+ 170us) server_negotiation.cc:299] Negotiation successful
0812 06:54:58.242970 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":370,"thread_start_us":130,"threads_started":1}
I20260812 06:54:58.243613 22212 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:54:58.236703 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43562 (local address 127.19.52.67:44177)
0812 06:54:58.237962 (+ 1259us) server_negotiation.cc:206] Beginning negotiation
0812 06:54:58.237975 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0812 06:54:58.237998 (+ 23us) server_negotiation.cc:407] Connection header received
0812 06:54:58.238076 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:54:58.238084 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:54:58.238311 (+ 227us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:54:58.238846 (+ 535us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 06:54:58.238976 (+ 130us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:54:58.239721 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:54:58.241263 (+ 1542us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:54:58.242182 (+ 919us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:54:58.242331 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:54:58.242524 (+ 193us) server_negotiation.cc:1035] Waiting for connection context
0812 06:54:58.243356 (+ 832us) server_negotiation.cc:299] Negotiation successful
0812 06:54:58.243457 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1156,"thread_start_us":122,"threads_started":1}
I20260812 06:54:58.244078 22214 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:54:58.237304 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.66:36983 (local address 127.0.0.1:34660)
0812 06:54:58.237534 (+ 230us) negotiation.cc:107] Waiting for socket to connect
0812 06:54:58.237547 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 06:54:58.237640 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:54:58.238665 (+ 1025us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:54:58.238668 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:54:58.238681 (+ 13us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 06:54:58.238767 (+ 86us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:54:58.238776 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:54:58.240377 (+ 1601us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:54:58.240381 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:54:58.242056 (+ 1675us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:54:58.242069 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:54:58.243629 (+ 1560us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:54:58.243636 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:54:58.243739 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:54:58.243775 (+ 36us) client_negotiation.cc:769] Sending connection context
0812 06:54:58.243858 (+ 83us) client_negotiation.cc:240] Negotiation successful
0812 06:54:58.243928 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":162,"thread_start_us":100,"threads_started":1}
I20260812 06:54:58.244745 22216 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:54:58.237732 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34660 (local address 127.19.52.66:36983)
0812 06:54:58.238199 (+ 467us) server_negotiation.cc:206] Beginning negotiation
0812 06:54:58.238211 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0812 06:54:58.238233 (+ 22us) server_negotiation.cc:407] Connection header received
0812 06:54:58.238308 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:54:58.238318 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:54:58.238500 (+ 182us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:54:58.238687 (+ 187us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 06:54:58.238875 (+ 188us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:54:58.240201 (+ 1326us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:54:58.242208 (+ 2007us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:54:58.243023 (+ 815us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:54:58.243608 (+ 585us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:54:58.244179 (+ 571us) server_negotiation.cc:1035] Waiting for connection context
0812 06:54:58.244228 (+ 49us) server_negotiation.cc:299] Negotiation successful
0812 06:54:58.244603 (+ 375us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":369,"thread_start_us":129,"threads_started":1}
I20260812 06:54:58.246534 21793 tablet_service.cc:1467] Processing CreateTablet for tablet 6e1ee82e0b0641ca8d086ef3f540c762 (DEFAULT_TABLE table=test-table [id=72072540752c4142b72c2c5fd18edc84]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 06:54:58.246774 22117 tablet_service.cc:1467] Processing CreateTablet for tablet 6e1ee82e0b0641ca8d086ef3f540c762 (DEFAULT_TABLE table=test-table [id=72072540752c4142b72c2c5fd18edc84]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 06:54:58.247694 21793 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6e1ee82e0b0641ca8d086ef3f540c762. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 06:54:58.247380 21953 tablet_service.cc:1467] Processing CreateTablet for tablet 6e1ee82e0b0641ca8d086ef3f540c762 (DEFAULT_TABLE table=test-table [id=72072540752c4142b72c2c5fd18edc84]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 06:54:58.247921 22117 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6e1ee82e0b0641ca8d086ef3f540c762. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 06:54:58.248378 21953 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6e1ee82e0b0641ca8d086ef3f540c762. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 06:54:58.254294 22217 tablet_bootstrap.cc:492] T 6e1ee82e0b0641ca8d086ef3f540c762 P cf0e18c976a24d659861decac8e6b157: Bootstrap starting.
I20260812 06:54:58.256189 22218 tablet_bootstrap.cc:492] T 6e1ee82e0b0641ca8d086ef3f540c762 P 6a49c31e22d740fab75a5123639a3502: Bootstrap starting.
I20260812 06:54:58.256641 22217 tablet_bootstrap.cc:654] T 6e1ee82e0b0641ca8d086ef3f540c762 P cf0e18c976a24d659861decac8e6b157: Neither blocks nor log segments found. Creating new log.
I20260812 06:54:58.257090 22219 tablet_bootstrap.cc:492] T 6e1ee82e0b0641ca8d086ef3f540c762 P adadc55f535f4b0dbe3d04e41251d456: Bootstrap starting.
I20260812 06:54:58.257652 22217 log.cc:824] T 6e1ee82e0b0641ca8d086ef3f540c762 P cf0e18c976a24d659861decac8e6b157: Log is configured to *not* fsync() on all Append() calls
I20260812 06:54:58.258805 22218 tablet_bootstrap.cc:654] T 6e1ee82e0b0641ca8d086ef3f540c762 P 6a49c31e22d740fab75a5123639a3502: Neither blocks nor log segments found. Creating new log.
I20260812 06:54:58.259425 22219 tablet_bootstrap.cc:654] T 6e1ee82e0b0641ca8d086ef3f540c762 P adadc55f535f4b0dbe3d04e41251d456: Neither blocks nor log segments found. Creating new log.
I20260812 06:54:58.259639 22218 log.cc:824] T 6e1ee82e0b0641ca8d086ef3f540c762 P 6a49c31e22d740fab75a5123639a3502: Log is configured to *not* fsync() on all Append() calls
I20260812 06:54:58.260308 22217 tablet_bootstrap.cc:492] T 6e1ee82e0b0641ca8d086ef3f540c762 P cf0e18c976a24d659861decac8e6b157: No bootstrap required, opened a new log
I20260812 06:54:58.260305 22219 log.cc:824] T 6e1ee82e0b0641ca8d086ef3f540c762 P adadc55f535f4b0dbe3d04e41251d456: Log is configured to *not* fsync() on all Append() calls
I20260812 06:54:58.260536 22217 ts_tablet_manager.cc:1397] T 6e1ee82e0b0641ca8d086ef3f540c762 P cf0e18c976a24d659861decac8e6b157: Time spent bootstrapping tablet: real 0.006s user 0.002s sys 0.002s
I20260812 06:54:58.261790 22218 tablet_bootstrap.cc:492] T 6e1ee82e0b0641ca8d086ef3f540c762 P 6a49c31e22d740fab75a5123639a3502: No bootstrap required, opened a new log
I20260812 06:54:58.261976 22218 ts_tablet_manager.cc:1397] T 6e1ee82e0b0641ca8d086ef3f540c762 P 6a49c31e22d740fab75a5123639a3502: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260812 06:54:58.262300 22219 tablet_bootstrap.cc:492] T 6e1ee82e0b0641ca8d086ef3f540c762 P adadc55f535f4b0dbe3d04e41251d456: No bootstrap required, opened a new log
I20260812 06:54:58.262547 22219 ts_tablet_manager.cc:1397] T 6e1ee82e0b0641ca8d086ef3f540c762 P adadc55f535f4b0dbe3d04e41251d456: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260812 06:54:58.265043 22217 raft_consensus.cc:348] T 6e1ee82e0b0641ca8d086ef3f540c762 P cf0e18c976a24d659861decac8e6b157 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a49c31e22d740fab75a5123639a3502" member_type: VOTER last_known_addr { host: "127.19.52.67" port: 44177 } } peers { permanent_uuid: "adadc55f535f4b0dbe3d04e41251d456" member_type: VOTER last_known_addr { host: "127.19.52.66" port: 36983 } } peers { permanent_uuid: "cf0e18c976a24d659861decac8e6b157" member_type: VOTER last_known_addr { host: "127.19.52.65" port: 45981 } }
I20260812 06:54:58.265348 22217 raft_consensus.cc:374] T 6e1ee82e0b0641ca8d086ef3f540c762 P cf0e18c976a24d659861decac8e6b157 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 06:54:58.265403 22217 raft_consensus.cc:729] T 6e1ee82e0b0641ca8d086ef3f540c762 P cf0e18c976a24d659861decac8e6b157 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cf0e18c976a24d659861decac8e6b157, State: Initialized, Role: FOLLOWER
I20260812 06:54:58.265815 22218 raft_consensus.cc:348] T 6e1ee82e0b0641ca8d086ef3f540c762 P 6a49c31e22d740fab75a5123639a3502 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a49c31e22d740fab75a5123639a3502" member_type: VOTER last_known_addr { host: "127.19.52.67" port: 44177 } } peers { permanent_uuid: "adadc55f535f4b0dbe3d04e41251d456" member_type: VOTER last_known_addr { host: "127.19.52.66" port: 36983 } } peers { permanent_uuid: "cf0e18c976a24d659861decac8e6b157" member_type: VOTER last_known_addr { host: "127.19.52.65" port: 45981 } }
I20260812 06:54:58.266062 22218 raft_consensus.cc:374] T 6e1ee82e0b0641ca8d086ef3f540c762 P 6a49c31e22d740fab75a5123639a3502 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 06:54:58.266122 22218 raft_consensus.cc:729] T 6e1ee82e0b0641ca8d086ef3f540c762 P 6a49c31e22d740fab75a5123639a3502 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6a49c31e22d740fab75a5123639a3502, State: Initialized, Role: FOLLOWER
I20260812 06:54:58.266186 22219 raft_consensus.cc:348] T 6e1ee82e0b0641ca8d086ef3f540c762 P adadc55f535f4b0dbe3d04e41251d456 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a49c31e22d740fab75a5123639a3502" member_type: VOTER last_known_addr { host: "127.19.52.67" port: 44177 } } peers { permanent_uuid: "adadc55f535f4b0dbe3d04e41251d456" member_type: VOTER last_known_addr { host: "127.19.52.66" port: 36983 } } peers { permanent_uuid: "cf0e18c976a24d659861decac8e6b157" member_type: VOTER last_known_addr { host: "127.19.52.65" port: 45981 } }
I20260812 06:54:58.266443 22219 raft_consensus.cc:374] T 6e1ee82e0b0641ca8d086ef3f540c762 P adadc55f535f4b0dbe3d04e41251d456 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 06:54:58.266531 22219 raft_consensus.cc:729] T 6e1ee82e0b0641ca8d086ef3f540c762 P adadc55f535f4b0dbe3d04e41251d456 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: adadc55f535f4b0dbe3d04e41251d456, State: Initialized, Role: FOLLOWER
I20260812 06:54:58.266554 22218 consensus_queue.cc:260] T 6e1ee82e0b0641ca8d086ef3f540c762 P 6a49c31e22d740fab75a5123639a3502 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a49c31e22d740fab75a5123639a3502" member_type: VOTER last_known_addr { host: "127.19.52.67" port: 44177 } } peers { permanent_uuid: "adadc55f535f4b0dbe3d04e41251d456" member_type: VOTER last_known_addr { host: "127.19.52.66" port: 36983 } } peers { permanent_uuid: "cf0e18c976a24d659861decac8e6b157" member_type: VOTER last_known_addr { host: "127.19.52.65" port: 45981 } }
I20260812 06:54:58.266554 22217 consensus_queue.cc:260] T 6e1ee82e0b0641ca8d086ef3f540c762 P cf0e18c976a24d659861decac8e6b157 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a49c31e22d740fab75a5123639a3502" member_type: VOTER last_known_addr { host: "127.19.52.67" port: 44177 } } peers { permanent_uuid: "adadc55f535f4b0dbe3d04e41251d456" member_type: VOTER last_known_addr { host: "127.19.52.66" port: 36983 } } peers { permanent_uuid: "cf0e18c976a24d659861decac8e6b157" member_type: VOTER last_known_addr { host: "127.19.52.65" port: 45981 } }
I20260812 06:54:58.267035 22219 consensus_queue.cc:260] T 6e1ee82e0b0641ca8d086ef3f540c762 P adadc55f535f4b0dbe3d04e41251d456 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a49c31e22d740fab75a5123639a3502" member_type: VOTER last_known_addr { host: "127.19.52.67" port: 44177 } } peers { permanent_uuid: "adadc55f535f4b0dbe3d04e41251d456" member_type: VOTER last_known_addr { host: "127.19.52.66" port: 36983 } } peers { permanent_uuid: "cf0e18c976a24d659861decac8e6b157" member_type: VOTER last_known_addr { host: "127.19.52.65" port: 45981 } }
I20260812 06:54:58.267269 21869 heartbeater.cc:503] Master 127.19.52.126:43141 was elected leader, sending a full tablet report...
I20260812 06:54:58.267617 22218 ts_tablet_manager.cc:1428] T 6e1ee82e0b0641ca8d086ef3f540c762 P 6a49c31e22d740fab75a5123639a3502: Time spent starting tablet: real 0.006s user 0.001s sys 0.005s
I20260812 06:54:58.267617 22217 ts_tablet_manager.cc:1428] T 6e1ee82e0b0641ca8d086ef3f540c762 P cf0e18c976a24d659861decac8e6b157: Time spent starting tablet: real 0.007s user 0.005s sys 0.001s
I20260812 06:54:58.267936 22193 heartbeater.cc:503] Master 127.19.52.126:43141 was elected leader, sending a full tablet report...
I20260812 06:54:58.268080 22219 ts_tablet_manager.cc:1428] T 6e1ee82e0b0641ca8d086ef3f540c762 P adadc55f535f4b0dbe3d04e41251d456: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260812 06:54:58.268436 22035 heartbeater.cc:503] Master 127.19.52.126:43141 was elected leader, sending a full tablet report...
I20260812 06:54:58.329824 22224 raft_consensus.cc:482] T 6e1ee82e0b0641ca8d086ef3f540c762 P cf0e18c976a24d659861decac8e6b157 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 06:54:58.330132 22224 raft_consensus.cc:504] T 6e1ee82e0b0641ca8d086ef3f540c762 P cf0e18c976a24d659861decac8e6b157 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a49c31e22d740fab75a5123639a3502" member_type: VOTER last_known_addr { host: "127.19.52.67" port: 44177 } } peers { permanent_uuid: "adadc55f535f4b0dbe3d04e41251d456" member_type: VOTER last_known_addr { host: "127.19.52.66" port: 36983 } } peers { permanent_uuid: "cf0e18c976a24d659861decac8e6b157" member_type: VOTER last_known_addr { host: "127.19.52.65" port: 45981 } }
I20260812 06:54:58.331367 22224 leader_election.cc:288] T 6e1ee82e0b0641ca8d086ef3f540c762 P cf0e18c976a24d659861decac8e6b157 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6a49c31e22d740fab75a5123639a3502 (127.19.52.67:44177), adadc55f535f4b0dbe3d04e41251d456 (127.19.52.66:36983)
I20260812 06:54:58.337060 22212 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:54:58.332079 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.52.65:41001 (local address 127.19.52.67:44177)
0812 06:54:58.332277 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0812 06:54:58.332281 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 06:54:58.332341 (+ 60us) server_negotiation.cc:407] Connection header received
0812 06:54:58.332592 (+ 251us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:54:58.332596 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:54:58.332655 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:54:58.332773 (+ 118us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 06:54:58.333162 (+ 389us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:54:58.333898 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:54:58.335593 (+ 1695us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:54:58.336302 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:54:58.336440 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:54:58.336496 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0812 06:54:58.336826 (+ 330us) server_negotiation.cc:299] Negotiation successful
0812 06:54:58.336920 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":95}
I20260812 06:54:58.337059 22216 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:54:58.332532 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.52.65:33381 (local address 127.19.52.66:36983)
0812 06:54:58.332705 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0812 06:54:58.332711 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 06:54:58.332878 (+ 167us) server_negotiation.cc:407] Connection header received
0812 06:54:58.333003 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:54:58.333010 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:54:58.333057 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:54:58.333159 (+ 102us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 06:54:58.333503 (+ 344us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:54:58.334148 (+ 645us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:54:58.335599 (+ 1451us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:54:58.336302 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:54:58.336440 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:54:58.336513 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0812 06:54:58.336832 (+ 319us) server_negotiation.cc:299] Negotiation successful
0812 06:54:58.336927 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":92}
I20260812 06:54:58.337102 22228 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:54:58.332387 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.66:36983 (local address 127.19.52.65:33381)
0812 06:54:58.332775 (+ 388us) negotiation.cc:107] Waiting for socket to connect
0812 06:54:58.332788 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 06:54:58.332901 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:54:58.333184 (+ 283us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:54:58.333186 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:54:58.333205 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 06:54:58.333329 (+ 124us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:54:58.333334 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:54:58.334292 (+ 958us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:54:58.334296 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:54:58.335469 (+ 1173us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:54:58.335479 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:54:58.336479 (+ 1000us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:54:58.336483 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:54:58.336680 (+ 197us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:54:58.336708 (+ 28us) client_negotiation.cc:769] Sending connection context
0812 06:54:58.336813 (+ 105us) client_negotiation.cc:240] Negotiation successful
0812 06:54:58.336893 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":292,"mutex_wait_us":101,"spinlock_wait_cycles":3072,"thread_start_us":105,"threads_started":1}
I20260812 06:54:58.337123 22227 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:54:58.331840 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.67:44177 (local address 127.19.52.65:41001)
0812 06:54:58.332251 (+ 411us) negotiation.cc:107] Waiting for socket to connect
0812 06:54:58.332269 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0812 06:54:58.332403 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:54:58.332799 (+ 396us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:54:58.332804 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:54:58.332831 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 06:54:58.333038 (+ 207us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:54:58.333045 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:54:58.334003 (+ 958us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:54:58.334010 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:54:58.335419 (+ 1409us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:54:58.335432 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:54:58.336479 (+ 1047us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:54:58.336483 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:54:58.336668 (+ 185us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:54:58.336699 (+ 31us) client_negotiation.cc:769] Sending connection context
0812 06:54:58.336806 (+ 107us) client_negotiation.cc:240] Negotiation successful
0812 06:54:58.336891 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":324,"mutex_wait_us":7,"spinlock_wait_cycles":17280,"thread_start_us":203,"threads_started":1}
I20260812 06:54:58.338078 21981 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6e1ee82e0b0641ca8d086ef3f540c762" candidate_uuid: "cf0e18c976a24d659861decac8e6b157" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "adadc55f535f4b0dbe3d04e41251d456" is_pre_election: true
I20260812 06:54:58.338068 22138 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6e1ee82e0b0641ca8d086ef3f540c762" candidate_uuid: "cf0e18c976a24d659861decac8e6b157" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6a49c31e22d740fab75a5123639a3502" is_pre_election: true
I20260812 06:54:58.338400 21981 raft_consensus.cc:2454] T 6e1ee82e0b0641ca8d086ef3f540c762 P adadc55f535f4b0dbe3d04e41251d456 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cf0e18c976a24d659861decac8e6b157 in term 0.
I20260812 06:54:58.338418 22138 raft_consensus.cc:2454] T 6e1ee82e0b0641ca8d086ef3f540c762 P 6a49c31e22d740fab75a5123639a3502 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cf0e18c976a24d659861decac8e6b157 in term 0.
I20260812 06:54:58.338917 21741 leader_election.cc:302] T 6e1ee82e0b0641ca8d086ef3f540c762 P cf0e18c976a24d659861decac8e6b157 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: adadc55f535f4b0dbe3d04e41251d456, cf0e18c976a24d659861decac8e6b157; no voters:
I20260812 06:54:58.339260 22224 raft_consensus.cc:2781] T 6e1ee82e0b0641ca8d086ef3f540c762 P cf0e18c976a24d659861decac8e6b157 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 06:54:58.339345 22224 raft_consensus.cc:482] T 6e1ee82e0b0641ca8d086ef3f540c762 P cf0e18c976a24d659861decac8e6b157 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 06:54:58.339375 22224 raft_consensus.cc:3037] T 6e1ee82e0b0641ca8d086ef3f540c762 P cf0e18c976a24d659861decac8e6b157 [term 0 FOLLOWER]: Advancing to term 1
I20260812 06:54:58.340646 22224 raft_consensus.cc:504] T 6e1ee82e0b0641ca8d086ef3f540c762 P cf0e18c976a24d659861decac8e6b157 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a49c31e22d740fab75a5123639a3502" member_type: VOTER last_known_addr { host: "127.19.52.67" port: 44177 } } peers { permanent_uuid: "adadc55f535f4b0dbe3d04e41251d456" member_type: VOTER last_known_addr { host: "127.19.52.66" port: 36983 } } peers { permanent_uuid: "cf0e18c976a24d659861decac8e6b157" member_type: VOTER last_known_addr { host: "127.19.52.65" port: 45981 } }
I20260812 06:54:58.341058 22224 leader_election.cc:288] T 6e1ee82e0b0641ca8d086ef3f540c762 P cf0e18c976a24d659861decac8e6b157 [CANDIDATE]: Term 1 election: Requested vote from peers 6a49c31e22d740fab75a5123639a3502 (127.19.52.67:44177), adadc55f535f4b0dbe3d04e41251d456 (127.19.52.66:36983)
I20260812 06:54:58.341542 22138 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6e1ee82e0b0641ca8d086ef3f540c762" candidate_uuid: "cf0e18c976a24d659861decac8e6b157" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6a49c31e22d740fab75a5123639a3502"
I20260812 06:54:58.341632 21981 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6e1ee82e0b0641ca8d086ef3f540c762" candidate_uuid: "cf0e18c976a24d659861decac8e6b157" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "adadc55f535f4b0dbe3d04e41251d456"
I20260812 06:54:58.341676 22138 raft_consensus.cc:3037] T 6e1ee82e0b0641ca8d086ef3f540c762 P 6a49c31e22d740fab75a5123639a3502 [term 0 FOLLOWER]: Advancing to term 1
I20260812 06:54:58.341812 21981 raft_consensus.cc:3037] T 6e1ee82e0b0641ca8d086ef3f540c762 P adadc55f535f4b0dbe3d04e41251d456 [term 0 FOLLOWER]: Advancing to term 1
I20260812 06:54:58.342818 22138 raft_consensus.cc:2454] T 6e1ee82e0b0641ca8d086ef3f540c762 P 6a49c31e22d740fab75a5123639a3502 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cf0e18c976a24d659861decac8e6b157 in term 1.
I20260812 06:54:58.342906 21981 raft_consensus.cc:2454] T 6e1ee82e0b0641ca8d086ef3f540c762 P adadc55f535f4b0dbe3d04e41251d456 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cf0e18c976a24d659861decac8e6b157 in term 1.
I20260812 06:54:58.343151 21741 leader_election.cc:302] T 6e1ee82e0b0641ca8d086ef3f540c762 P cf0e18c976a24d659861decac8e6b157 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a49c31e22d740fab75a5123639a3502, cf0e18c976a24d659861decac8e6b157; no voters:
I20260812 06:54:58.343396 22224 raft_consensus.cc:2781] T 6e1ee82e0b0641ca8d086ef3f540c762 P cf0e18c976a24d659861decac8e6b157 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 06:54:58.343689 22224 raft_consensus.cc:686] T 6e1ee82e0b0641ca8d086ef3f540c762 P cf0e18c976a24d659861decac8e6b157 [term 1 LEADER]: Becoming Leader. State: Replica: cf0e18c976a24d659861decac8e6b157, State: Running, Role: LEADER
I20260812 06:54:58.344043 22224 consensus_queue.cc:237] T 6e1ee82e0b0641ca8d086ef3f540c762 P cf0e18c976a24d659861decac8e6b157 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a49c31e22d740fab75a5123639a3502" member_type: VOTER last_known_addr { host: "127.19.52.67" port: 44177 } } peers { permanent_uuid: "adadc55f535f4b0dbe3d04e41251d456" member_type: VOTER last_known_addr { host: "127.19.52.66" port: 36983 } } peers { permanent_uuid: "cf0e18c976a24d659861decac8e6b157" member_type: VOTER last_known_addr { host: "127.19.52.65" port: 45981 } }
I20260812 06:54:58.347754 21632 catalog_manager.cc:5495] T 6e1ee82e0b0641ca8d086ef3f540c762 P cf0e18c976a24d659861decac8e6b157 reported cstate change: term changed from 0 to 1, leader changed from <none> to cf0e18c976a24d659861decac8e6b157 (127.19.52.65). New cstate: current_term: 1 leader_uuid: "cf0e18c976a24d659861decac8e6b157" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a49c31e22d740fab75a5123639a3502" member_type: VOTER last_known_addr { host: "127.19.52.67" port: 44177 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "adadc55f535f4b0dbe3d04e41251d456" member_type: VOTER last_known_addr { host: "127.19.52.66" port: 36983 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cf0e18c976a24d659861decac8e6b157" member_type: VOTER last_known_addr { host: "127.19.52.65" port: 45981 } health_report { overall_health: HEALTHY } } }
Aug 12 06:54:58 dist-test-slave-65mx krb5kdc[21585](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517697, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.52.126@KRBTEST.COM for kudu/127.19.52.126@KRBTEST.COM
I20260812 06:54:58.392346 22238 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:54:58.382557 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.126:43141 (local address 127.0.0.1:46628)
0812 06:54:58.382874 (+ 317us) negotiation.cc:107] Waiting for socket to connect
0812 06:54:58.382892 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0812 06:54:58.383014 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:54:58.383385 (+ 371us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:54:58.383387 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:54:58.383797 (+ 410us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:54:58.383899 (+ 102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:54:58.383905 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:54:58.384849 (+ 944us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:54:58.384852 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:54:58.385472 (+ 620us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:54:58.385481 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:54:58.386034 (+ 553us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:54:58.386039 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:54:58.386135 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:54:58.386865 (+ 730us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 06:54:58.386888 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:54:58.388478 (+ 1590us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 06:54:58.390566 (+ 2088us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:54:58.390576 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:54:58.390581 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:54:58.390849 (+ 268us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:54:58.391134 (+ 285us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:54:58.391136 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:54:58.391137 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:54:58.391221 (+ 84us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:54:58.391584 (+ 363us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 06:54:58.391591 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 06:54:58.391752 (+ 161us) client_negotiation.cc:769] Sending connection context
0812 06:54:58.391896 (+ 144us) client_negotiation.cc:240] Negotiation successful
0812 06:54:58.392045 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":229,"thread_start_us":137,"threads_started":1}
I20260812 06:54:58.392513 21693 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:54:58.382602 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46628 (local address 127.19.52.126:43141)
0812 06:54:58.382767 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0812 06:54:58.382771 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 06:54:58.383162 (+ 391us) server_negotiation.cc:407] Connection header received
0812 06:54:58.383214 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:54:58.383216 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:54:58.383261 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:54:58.383347 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:54:58.384052 (+ 705us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:54:58.384740 (+ 688us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:54:58.385593 (+ 853us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:54:58.385810 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:54:58.385903 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:54:58.388716 (+ 2813us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:54:58.388724 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:54:58.388728 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:54:58.388759 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:54:58.390452 (+ 1693us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:54:58.390959 (+ 507us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:54:58.390962 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:54:58.390963 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:54:58.391045 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:54:58.391302 (+ 257us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:54:58.391304 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:54:58.391305 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:54:58.391489 (+ 184us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:54:58.391566 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0812 06:54:58.391970 (+ 404us) server_negotiation.cc:299] Negotiation successful
0812 06:54:58.392108 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":24,"server-negotiator.queue_time_us":78}
I20260812 06:54:58.396864 21632 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.19.52.126@KRBTEST.COM'} at 127.0.0.1:46628:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260812 06:54:58.397657 21632 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 06:54:58.402705 21793 tablet_service.cc:1467] Processing CreateTablet for tablet 15e2ce7e114f46d08df283e2a5b670d7 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=5c09926a4a514ec19174276191e79e4e]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260812 06:54:58.403432 21793 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 15e2ce7e114f46d08df283e2a5b670d7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 06:54:58.403123 22117 tablet_service.cc:1467] Processing CreateTablet for tablet 15e2ce7e114f46d08df283e2a5b670d7 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=5c09926a4a514ec19174276191e79e4e]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260812 06:54:58.403123 21953 tablet_service.cc:1467] Processing CreateTablet for tablet 15e2ce7e114f46d08df283e2a5b670d7 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=5c09926a4a514ec19174276191e79e4e]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260812 06:54:58.403823 21953 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 15e2ce7e114f46d08df283e2a5b670d7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 06:54:58.403823 22117 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 15e2ce7e114f46d08df283e2a5b670d7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 06:54:58.406343 22217 tablet_bootstrap.cc:492] T 15e2ce7e114f46d08df283e2a5b670d7 P cf0e18c976a24d659861decac8e6b157: Bootstrap starting.
I20260812 06:54:58.406860 22219 tablet_bootstrap.cc:492] T 15e2ce7e114f46d08df283e2a5b670d7 P adadc55f535f4b0dbe3d04e41251d456: Bootstrap starting.
I20260812 06:54:58.407382 22217 tablet_bootstrap.cc:654] T 15e2ce7e114f46d08df283e2a5b670d7 P cf0e18c976a24d659861decac8e6b157: Neither blocks nor log segments found. Creating new log.
I20260812 06:54:58.407828 22219 tablet_bootstrap.cc:654] T 15e2ce7e114f46d08df283e2a5b670d7 P adadc55f535f4b0dbe3d04e41251d456: Neither blocks nor log segments found. Creating new log.
I20260812 06:54:58.408775 22218 tablet_bootstrap.cc:492] T 15e2ce7e114f46d08df283e2a5b670d7 P 6a49c31e22d740fab75a5123639a3502: Bootstrap starting.
I20260812 06:54:58.408973 22217 tablet_bootstrap.cc:492] T 15e2ce7e114f46d08df283e2a5b670d7 P cf0e18c976a24d659861decac8e6b157: No bootstrap required, opened a new log
I20260812 06:54:58.409066 22217 ts_tablet_manager.cc:1397] T 15e2ce7e114f46d08df283e2a5b670d7 P cf0e18c976a24d659861decac8e6b157: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.001s
I20260812 06:54:58.409593 22217 raft_consensus.cc:348] T 15e2ce7e114f46d08df283e2a5b670d7 P cf0e18c976a24d659861decac8e6b157 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf0e18c976a24d659861decac8e6b157" member_type: VOTER last_known_addr { host: "127.19.52.65" port: 45981 } } peers { permanent_uuid: "adadc55f535f4b0dbe3d04e41251d456" member_type: VOTER last_known_addr { host: "127.19.52.66" port: 36983 } } peers { permanent_uuid: "6a49c31e22d740fab75a5123639a3502" member_type: VOTER last_known_addr { host: "127.19.52.67" port: 44177 } }
I20260812 06:54:58.409781 22218 tablet_bootstrap.cc:654] T 15e2ce7e114f46d08df283e2a5b670d7 P 6a49c31e22d740fab75a5123639a3502: Neither blocks nor log segments found. Creating new log.
I20260812 06:54:58.409777 22217 raft_consensus.cc:374] T 15e2ce7e114f46d08df283e2a5b670d7 P cf0e18c976a24d659861decac8e6b157 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 06:54:58.409816 22217 raft_consensus.cc:729] T 15e2ce7e114f46d08df283e2a5b670d7 P cf0e18c976a24d659861decac8e6b157 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cf0e18c976a24d659861decac8e6b157, State: Initialized, Role: FOLLOWER
I20260812 06:54:58.409976 22217 consensus_queue.cc:260] T 15e2ce7e114f46d08df283e2a5b670d7 P cf0e18c976a24d659861decac8e6b157 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf0e18c976a24d659861decac8e6b157" member_type: VOTER last_known_addr { host: "127.19.52.65" port: 45981 } } peers { permanent_uuid: "adadc55f535f4b0dbe3d04e41251d456" member_type: VOTER last_known_addr { host: "127.19.52.66" port: 36983 } } peers { permanent_uuid: "6a49c31e22d740fab75a5123639a3502" member_type: VOTER last_known_addr { host: "127.19.52.67" port: 44177 } }
I20260812 06:54:58.410240 22224 tablet_replica.cc:439] T 15e2ce7e114f46d08df283e2a5b670d7 P cf0e18c976a24d659861decac8e6b157: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf0e18c976a24d659861decac8e6b157" member_type: VOTER last_known_addr { host: "127.19.52.65" port: 45981 } } peers { permanent_uuid: "adadc55f535f4b0dbe3d04e41251d456" member_type: VOTER last_known_addr { host: "127.19.52.66" port: 36983 } } peers { permanent_uuid: "6a49c31e22d740fab75a5123639a3502" member_type: VOTER last_known_addr { host: "127.19.52.67" port: 44177 } } }
I20260812 06:54:58.410387 22224 tablet_replica.cc:442] T 15e2ce7e114f46d08df283e2a5b670d7 P cf0e18c976a24d659861decac8e6b157: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 06:54:58.411044 22217 ts_tablet_manager.cc:1428] T 15e2ce7e114f46d08df283e2a5b670d7 P cf0e18c976a24d659861decac8e6b157: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260812 06:54:58.411244 22219 tablet_bootstrap.cc:492] T 15e2ce7e114f46d08df283e2a5b670d7 P adadc55f535f4b0dbe3d04e41251d456: No bootstrap required, opened a new log
I20260812 06:54:58.411319 22219 ts_tablet_manager.cc:1397] T 15e2ce7e114f46d08df283e2a5b670d7 P adadc55f535f4b0dbe3d04e41251d456: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260812 06:54:58.411743 22219 raft_consensus.cc:348] T 15e2ce7e114f46d08df283e2a5b670d7 P adadc55f535f4b0dbe3d04e41251d456 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf0e18c976a24d659861decac8e6b157" member_type: VOTER last_known_addr { host: "127.19.52.65" port: 45981 } } peers { permanent_uuid: "adadc55f535f4b0dbe3d04e41251d456" member_type: VOTER last_known_addr { host: "127.19.52.66" port: 36983 } } peers { permanent_uuid: "6a49c31e22d740fab75a5123639a3502" member_type: VOTER last_known_addr { host: "127.19.52.67" port: 44177 } }
I20260812 06:54:58.411839 22219 raft_consensus.cc:374] T 15e2ce7e114f46d08df283e2a5b670d7 P adadc55f535f4b0dbe3d04e41251d456 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 06:54:58.411865 22219 raft_consensus.cc:729] T 15e2ce7e114f46d08df283e2a5b670d7 P adadc55f535f4b0dbe3d04e41251d456 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: adadc55f535f4b0dbe3d04e41251d456, State: Initialized, Role: FOLLOWER
I20260812 06:54:58.411984 22219 consensus_queue.cc:260] T 15e2ce7e114f46d08df283e2a5b670d7 P adadc55f535f4b0dbe3d04e41251d456 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf0e18c976a24d659861decac8e6b157" member_type: VOTER last_known_addr { host: "127.19.52.65" port: 45981 } } peers { permanent_uuid: "adadc55f535f4b0dbe3d04e41251d456" member_type: VOTER last_known_addr { host: "127.19.52.66" port: 36983 } } peers { permanent_uuid: "6a49c31e22d740fab75a5123639a3502" member_type: VOTER last_known_addr { host: "127.19.52.67" port: 44177 } }
I20260812 06:54:58.412221 22219 ts_tablet_manager.cc:1428] T 15e2ce7e114f46d08df283e2a5b670d7 P adadc55f535f4b0dbe3d04e41251d456: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260812 06:54:58.412271 22226 tablet_replica.cc:439] T 15e2ce7e114f46d08df283e2a5b670d7 P adadc55f535f4b0dbe3d04e41251d456: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf0e18c976a24d659861decac8e6b157" member_type: VOTER last_known_addr { host: "127.19.52.65" port: 45981 } } peers { permanent_uuid: "adadc55f535f4b0dbe3d04e41251d456" member_type: VOTER last_known_addr { host: "127.19.52.66" port: 36983 } } peers { permanent_uuid: "6a49c31e22d740fab75a5123639a3502" member_type: VOTER last_known_addr { host: "127.19.52.67" port: 44177 } } }
I20260812 06:54:58.412444 22226 tablet_replica.cc:442] T 15e2ce7e114f46d08df283e2a5b670d7 P adadc55f535f4b0dbe3d04e41251d456: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 06:54:58.413271 22218 tablet_bootstrap.cc:492] T 15e2ce7e114f46d08df283e2a5b670d7 P 6a49c31e22d740fab75a5123639a3502: No bootstrap required, opened a new log
I20260812 06:54:58.413372 22218 ts_tablet_manager.cc:1397] T 15e2ce7e114f46d08df283e2a5b670d7 P 6a49c31e22d740fab75a5123639a3502: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.001s
I20260812 06:54:58.413776 22218 raft_consensus.cc:348] T 15e2ce7e114f46d08df283e2a5b670d7 P 6a49c31e22d740fab75a5123639a3502 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf0e18c976a24d659861decac8e6b157" member_type: VOTER last_known_addr { host: "127.19.52.65" port: 45981 } } peers { permanent_uuid: "adadc55f535f4b0dbe3d04e41251d456" member_type: VOTER last_known_addr { host: "127.19.52.66" port: 36983 } } peers { permanent_uuid: "6a49c31e22d740fab75a5123639a3502" member_type: VOTER last_known_addr { host: "127.19.52.67" port: 44177 } }
I20260812 06:54:58.413913 22218 raft_consensus.cc:374] T 15e2ce7e114f46d08df283e2a5b670d7 P 6a49c31e22d740fab75a5123639a3502 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 06:54:58.413954 22218 raft_consensus.cc:729] T 15e2ce7e114f46d08df283e2a5b670d7 P 6a49c31e22d740fab75a5123639a3502 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6a49c31e22d740fab75a5123639a3502, State: Initialized, Role: FOLLOWER
I20260812 06:54:58.414091 22218 consensus_queue.cc:260] T 15e2ce7e114f46d08df283e2a5b670d7 P 6a49c31e22d740fab75a5123639a3502 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf0e18c976a24d659861decac8e6b157" member_type: VOTER last_known_addr { host: "127.19.52.65" port: 45981 } } peers { permanent_uuid: "adadc55f535f4b0dbe3d04e41251d456" member_type: VOTER last_known_addr { host: "127.19.52.66" port: 36983 } } peers { permanent_uuid: "6a49c31e22d740fab75a5123639a3502" member_type: VOTER last_known_addr { host: "127.19.52.67" port: 44177 } }
I20260812 06:54:58.414413 22218 ts_tablet_manager.cc:1428] T 15e2ce7e114f46d08df283e2a5b670d7 P 6a49c31e22d740fab75a5123639a3502: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260812 06:54:58.414382 22225 tablet_replica.cc:439] T 15e2ce7e114f46d08df283e2a5b670d7 P 6a49c31e22d740fab75a5123639a3502: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf0e18c976a24d659861decac8e6b157" member_type: VOTER last_known_addr { host: "127.19.52.65" port: 45981 } } peers { permanent_uuid: "adadc55f535f4b0dbe3d04e41251d456" member_type: VOTER last_known_addr { host: "127.19.52.66" port: 36983 } } peers { permanent_uuid: "6a49c31e22d740fab75a5123639a3502" member_type: VOTER last_known_addr { host: "127.19.52.67" port: 44177 } } }
I20260812 06:54:58.414521 22225 tablet_replica.cc:442] T 15e2ce7e114f46d08df283e2a5b670d7 P 6a49c31e22d740fab75a5123639a3502: This TxnStatusTablet replica's current role is: FOLLOWER
W20260812 06:54:58.431672 22195 tablet.cc:2369] T 15e2ce7e114f46d08df283e2a5b670d7 P 6a49c31e22d740fab75a5123639a3502: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 06:54:58.458370 22036 tablet.cc:2369] T 15e2ce7e114f46d08df283e2a5b670d7 P adadc55f535f4b0dbe3d04e41251d456: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 06:54:58.474448 21870 tablet.cc:2369] T 15e2ce7e114f46d08df283e2a5b670d7 P cf0e18c976a24d659861decac8e6b157: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 06:54:58.790223 22224 consensus_queue.cc:1035] T 6e1ee82e0b0641ca8d086ef3f540c762 P cf0e18c976a24d659861decac8e6b157 [LEADER]: Connected to new peer: Peer: permanent_uuid: "adadc55f535f4b0dbe3d04e41251d456" member_type: VOTER last_known_addr { host: "127.19.52.66" port: 36983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 06:54:58.793148 22224 raft_consensus.cc:482] T 15e2ce7e114f46d08df283e2a5b670d7 P cf0e18c976a24d659861decac8e6b157 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 06:54:58.793306 22224 raft_consensus.cc:504] T 15e2ce7e114f46d08df283e2a5b670d7 P cf0e18c976a24d659861decac8e6b157 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf0e18c976a24d659861decac8e6b157" member_type: VOTER last_known_addr { host: "127.19.52.65" port: 45981 } } peers { permanent_uuid: "adadc55f535f4b0dbe3d04e41251d456" member_type: VOTER last_known_addr { host: "127.19.52.66" port: 36983 } } peers { permanent_uuid: "6a49c31e22d740fab75a5123639a3502" member_type: VOTER last_known_addr { host: "127.19.52.67" port: 44177 } }
I20260812 06:54:58.793740 22224 leader_election.cc:288] T 15e2ce7e114f46d08df283e2a5b670d7 P cf0e18c976a24d659861decac8e6b157 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers adadc55f535f4b0dbe3d04e41251d456 (127.19.52.66:36983), 6a49c31e22d740fab75a5123639a3502 (127.19.52.67:44177)
I20260812 06:54:58.794111 21980 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "15e2ce7e114f46d08df283e2a5b670d7" candidate_uuid: "cf0e18c976a24d659861decac8e6b157" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "adadc55f535f4b0dbe3d04e41251d456" is_pre_election: true
I20260812 06:54:58.794220 22138 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "15e2ce7e114f46d08df283e2a5b670d7" candidate_uuid: "cf0e18c976a24d659861decac8e6b157" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6a49c31e22d740fab75a5123639a3502" is_pre_election: true
I20260812 06:54:58.794274 21980 raft_consensus.cc:2454] T 15e2ce7e114f46d08df283e2a5b670d7 P adadc55f535f4b0dbe3d04e41251d456 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cf0e18c976a24d659861decac8e6b157 in term 0.
I20260812 06:54:58.794364 22138 raft_consensus.cc:2454] T 15e2ce7e114f46d08df283e2a5b670d7 P 6a49c31e22d740fab75a5123639a3502 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cf0e18c976a24d659861decac8e6b157 in term 0.
I20260812 06:54:58.794764 21741 leader_election.cc:302] T 15e2ce7e114f46d08df283e2a5b670d7 P cf0e18c976a24d659861decac8e6b157 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: adadc55f535f4b0dbe3d04e41251d456, cf0e18c976a24d659861decac8e6b157; no voters:
I20260812 06:54:58.795001 22229 raft_consensus.cc:2781] T 15e2ce7e114f46d08df283e2a5b670d7 P cf0e18c976a24d659861decac8e6b157 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 06:54:58.795097 22229 raft_consensus.cc:482] T 15e2ce7e114f46d08df283e2a5b670d7 P cf0e18c976a24d659861decac8e6b157 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 06:54:58.795169 22229 raft_consensus.cc:3037] T 15e2ce7e114f46d08df283e2a5b670d7 P cf0e18c976a24d659861decac8e6b157 [term 0 FOLLOWER]: Advancing to term 1
I20260812 06:54:58.796648 22229 raft_consensus.cc:504] T 15e2ce7e114f46d08df283e2a5b670d7 P cf0e18c976a24d659861decac8e6b157 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf0e18c976a24d659861decac8e6b157" member_type: VOTER last_known_addr { host: "127.19.52.65" port: 45981 } } peers { permanent_uuid: "adadc55f535f4b0dbe3d04e41251d456" member_type: VOTER last_known_addr { host: "127.19.52.66" port: 36983 } } peers { permanent_uuid: "6a49c31e22d740fab75a5123639a3502" member_type: VOTER last_known_addr { host: "127.19.52.67" port: 44177 } }
I20260812 06:54:58.797053 22229 leader_election.cc:288] T 15e2ce7e114f46d08df283e2a5b670d7 P cf0e18c976a24d659861decac8e6b157 [CANDIDATE]: Term 1 election: Requested vote from peers adadc55f535f4b0dbe3d04e41251d456 (127.19.52.66:36983), 6a49c31e22d740fab75a5123639a3502 (127.19.52.67:44177)
I20260812 06:54:58.797538 21980 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "15e2ce7e114f46d08df283e2a5b670d7" candidate_uuid: "cf0e18c976a24d659861decac8e6b157" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "adadc55f535f4b0dbe3d04e41251d456"
I20260812 06:54:58.797684 21980 raft_consensus.cc:3037] T 15e2ce7e114f46d08df283e2a5b670d7 P adadc55f535f4b0dbe3d04e41251d456 [term 0 FOLLOWER]: Advancing to term 1
I20260812 06:54:58.797729 22140 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "15e2ce7e114f46d08df283e2a5b670d7" candidate_uuid: "cf0e18c976a24d659861decac8e6b157" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6a49c31e22d740fab75a5123639a3502"
I20260812 06:54:58.797860 22140 raft_consensus.cc:3037] T 15e2ce7e114f46d08df283e2a5b670d7 P 6a49c31e22d740fab75a5123639a3502 [term 0 FOLLOWER]: Advancing to term 1
I20260812 06:54:58.799057 21980 raft_consensus.cc:2454] T 15e2ce7e114f46d08df283e2a5b670d7 P adadc55f535f4b0dbe3d04e41251d456 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cf0e18c976a24d659861decac8e6b157 in term 1.
I20260812 06:54:58.799165 22140 raft_consensus.cc:2454] T 15e2ce7e114f46d08df283e2a5b670d7 P 6a49c31e22d740fab75a5123639a3502 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cf0e18c976a24d659861decac8e6b157 in term 1.
I20260812 06:54:58.799383 21741 leader_election.cc:302] T 15e2ce7e114f46d08df283e2a5b670d7 P cf0e18c976a24d659861decac8e6b157 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: adadc55f535f4b0dbe3d04e41251d456, cf0e18c976a24d659861decac8e6b157; no voters:
I20260812 06:54:58.799558 22246 raft_consensus.cc:2781] T 15e2ce7e114f46d08df283e2a5b670d7 P cf0e18c976a24d659861decac8e6b157 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 06:54:58.799671 22246 raft_consensus.cc:686] T 15e2ce7e114f46d08df283e2a5b670d7 P cf0e18c976a24d659861decac8e6b157 [term 1 LEADER]: Becoming Leader. State: Replica: cf0e18c976a24d659861decac8e6b157, State: Running, Role: LEADER
I20260812 06:54:58.799764 22243 consensus_queue.cc:1035] T 6e1ee82e0b0641ca8d086ef3f540c762 P cf0e18c976a24d659861decac8e6b157 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a49c31e22d740fab75a5123639a3502" member_type: VOTER last_known_addr { host: "127.19.52.67" port: 44177 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 06:54:58.799820 22246 consensus_queue.cc:237] T 15e2ce7e114f46d08df283e2a5b670d7 P cf0e18c976a24d659861decac8e6b157 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf0e18c976a24d659861decac8e6b157" member_type: VOTER last_known_addr { host: "127.19.52.65" port: 45981 } } peers { permanent_uuid: "adadc55f535f4b0dbe3d04e41251d456" member_type: VOTER last_known_addr { host: "127.19.52.66" port: 36983 } } peers { permanent_uuid: "6a49c31e22d740fab75a5123639a3502" member_type: VOTER last_known_addr { host: "127.19.52.67" port: 44177 } }
I20260812 06:54:58.800957 22229 tablet_replica.cc:439] T 15e2ce7e114f46d08df283e2a5b670d7 P cf0e18c976a24d659861decac8e6b157: TxnStatusTablet state changed. Reason: New leader cf0e18c976a24d659861decac8e6b157. Latest consensus state: current_term: 1 leader_uuid: "cf0e18c976a24d659861decac8e6b157" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf0e18c976a24d659861decac8e6b157" member_type: VOTER last_known_addr { host: "127.19.52.65" port: 45981 } } peers { permanent_uuid: "adadc55f535f4b0dbe3d04e41251d456" member_type: VOTER last_known_addr { host: "127.19.52.66" port: 36983 } } peers { permanent_uuid: "6a49c31e22d740fab75a5123639a3502" member_type: VOTER last_known_addr { host: "127.19.52.67" port: 44177 } } }
I20260812 06:54:58.801074 22229 tablet_replica.cc:442] T 15e2ce7e114f46d08df283e2a5b670d7 P cf0e18c976a24d659861decac8e6b157: This TxnStatusTablet replica's current role is: LEADER
I20260812 06:54:58.801515 22249 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260812 06:54:58.801770 22249 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260812 06:54:58.801759 21632 catalog_manager.cc:5495] T 15e2ce7e114f46d08df283e2a5b670d7 P cf0e18c976a24d659861decac8e6b157 reported cstate change: term changed from 0 to 1, leader changed from <none> to cf0e18c976a24d659861decac8e6b157 (127.19.52.65). New cstate: current_term: 1 leader_uuid: "cf0e18c976a24d659861decac8e6b157" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf0e18c976a24d659861decac8e6b157" member_type: VOTER last_known_addr { host: "127.19.52.65" port: 45981 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "adadc55f535f4b0dbe3d04e41251d456" member_type: VOTER last_known_addr { host: "127.19.52.66" port: 36983 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6a49c31e22d740fab75a5123639a3502" member_type: VOTER last_known_addr { host: "127.19.52.67" port: 44177 } health_report { overall_health: UNKNOWN } } }
I20260812 06:54:58.874010 22238 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:54:58.868841 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.65:45981 (local address 127.0.0.1:47232)
0812 06:54:58.869042 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0812 06:54:58.869062 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0812 06:54:58.869189 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:54:58.869657 (+ 468us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:54:58.869659 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:54:58.869667 (+ 8us) client_negotiation.cc:189] Negotiated authn=TOKEN
0812 06:54:58.869770 (+ 103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:54:58.869774 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:54:58.871100 (+ 1326us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:54:58.871103 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:54:58.871932 (+ 829us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:54:58.871944 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:54:58.872695 (+ 751us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:54:58.872698 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:54:58.872790 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:54:58.872847 (+ 57us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0812 06:54:58.873723 (+ 876us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0812 06:54:58.873739 (+ 16us) client_negotiation.cc:769] Sending connection context
0812 06:54:58.873801 (+ 62us) client_negotiation.cc:240] Negotiation successful
0812 06:54:58.873883 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":63}
I20260812 06:54:58.874079 22254 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:54:58.868981 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47232 (local address 127.19.52.65:45981)
0812 06:54:58.869296 (+ 315us) server_negotiation.cc:206] Beginning negotiation
0812 06:54:58.869301 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 06:54:58.869319 (+ 18us) server_negotiation.cc:407] Connection header received
0812 06:54:58.869411 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:54:58.869414 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:54:58.869501 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:54:58.869631 (+ 130us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 06:54:58.869896 (+ 265us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:54:58.870912 (+ 1016us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:54:58.872088 (+ 1176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:54:58.872468 (+ 380us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:54:58.872609 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:54:58.873015 (+ 406us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 06:54:58.873611 (+ 596us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 06:54:58.873727 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0812 06:54:58.873839 (+ 112us) server_negotiation.cc:299] Negotiation successful
0812 06:54:58.873908 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":210,"thread_start_us":94,"threads_started":1}
I20260812 06:54:58.879201 21980 raft_consensus.cc:1261] T 15e2ce7e114f46d08df283e2a5b670d7 P adadc55f535f4b0dbe3d04e41251d456 [term 1 FOLLOWER]: Refusing update from remote peer cf0e18c976a24d659861decac8e6b157: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 06:54:58.879194 22140 raft_consensus.cc:1261] T 15e2ce7e114f46d08df283e2a5b670d7 P 6a49c31e22d740fab75a5123639a3502 [term 1 FOLLOWER]: Refusing update from remote peer cf0e18c976a24d659861decac8e6b157: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 06:54:58.879652 22229 consensus_queue.cc:1035] T 15e2ce7e114f46d08df283e2a5b670d7 P cf0e18c976a24d659861decac8e6b157 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a49c31e22d740fab75a5123639a3502" member_type: VOTER last_known_addr { host: "127.19.52.67" port: 44177 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 06:54:58.879832 22246 consensus_queue.cc:1035] T 15e2ce7e114f46d08df283e2a5b670d7 P cf0e18c976a24d659861decac8e6b157 [LEADER]: Connected to new peer: Peer: permanent_uuid: "adadc55f535f4b0dbe3d04e41251d456" member_type: VOTER last_known_addr { host: "127.19.52.66" port: 36983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 06:54:58.882519 22225 tablet_replica.cc:439] T 15e2ce7e114f46d08df283e2a5b670d7 P 6a49c31e22d740fab75a5123639a3502: TxnStatusTablet state changed. Reason: New leader cf0e18c976a24d659861decac8e6b157. Latest consensus state: current_term: 1 leader_uuid: "cf0e18c976a24d659861decac8e6b157" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf0e18c976a24d659861decac8e6b157" member_type: VOTER last_known_addr { host: "127.19.52.65" port: 45981 } } peers { permanent_uuid: "adadc55f535f4b0dbe3d04e41251d456" member_type: VOTER last_known_addr { host: "127.19.52.66" port: 36983 } } peers { permanent_uuid: "6a49c31e22d740fab75a5123639a3502" member_type: VOTER last_known_addr { host: "127.19.52.67" port: 44177 } } }
I20260812 06:54:58.882647 22225 tablet_replica.cc:442] T 15e2ce7e114f46d08df283e2a5b670d7 P 6a49c31e22d740fab75a5123639a3502: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 06:54:58.884676 22229 tablet_replica.cc:439] T 15e2ce7e114f46d08df283e2a5b670d7 P cf0e18c976a24d659861decac8e6b157: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "cf0e18c976a24d659861decac8e6b157" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf0e18c976a24d659861decac8e6b157" member_type: VOTER last_known_addr { host: "127.19.52.65" port: 45981 } } peers { permanent_uuid: "adadc55f535f4b0dbe3d04e41251d456" member_type: VOTER last_known_addr { host: "127.19.52.66" port: 36983 } } peers { permanent_uuid: "6a49c31e22d740fab75a5123639a3502" member_type: VOTER last_known_addr { host: "127.19.52.67" port: 44177 } } }
I20260812 06:54:58.884814 22229 tablet_replica.cc:442] T 15e2ce7e114f46d08df283e2a5b670d7 P cf0e18c976a24d659861decac8e6b157: This TxnStatusTablet replica's current role is: LEADER
I20260812 06:54:58.885737 22226 tablet_replica.cc:439] T 15e2ce7e114f46d08df283e2a5b670d7 P adadc55f535f4b0dbe3d04e41251d456: TxnStatusTablet state changed. Reason: New leader cf0e18c976a24d659861decac8e6b157. Latest consensus state: current_term: 1 leader_uuid: "cf0e18c976a24d659861decac8e6b157" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf0e18c976a24d659861decac8e6b157" member_type: VOTER last_known_addr { host: "127.19.52.65" port: 45981 } } peers { permanent_uuid: "adadc55f535f4b0dbe3d04e41251d456" member_type: VOTER last_known_addr { host: "127.19.52.66" port: 36983 } } peers { permanent_uuid: "6a49c31e22d740fab75a5123639a3502" member_type: VOTER last_known_addr { host: "127.19.52.67" port: 44177 } } }
I20260812 06:54:58.885875 22226 tablet_replica.cc:442] T 15e2ce7e114f46d08df283e2a5b670d7 P adadc55f535f4b0dbe3d04e41251d456: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 06:54:58.886044 22244 mvcc.cc:205] Tried to move back new op lower bound from 7317576494603509760 to 7317576494285987840. Current Snapshot: MvccSnapshot[applied={T|T < 7317576494603509760}]
I20260812 06:54:58.887517 22226 tablet_replica.cc:439] T 15e2ce7e114f46d08df283e2a5b670d7 P adadc55f535f4b0dbe3d04e41251d456: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "cf0e18c976a24d659861decac8e6b157" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf0e18c976a24d659861decac8e6b157" member_type: VOTER last_known_addr { host: "127.19.52.65" port: 45981 } } peers { permanent_uuid: "adadc55f535f4b0dbe3d04e41251d456" member_type: VOTER last_known_addr { host: "127.19.52.66" port: 36983 } } peers { permanent_uuid: "6a49c31e22d740fab75a5123639a3502" member_type: VOTER last_known_addr { host: "127.19.52.67" port: 44177 } } }
I20260812 06:54:58.887643 22226 tablet_replica.cc:442] T 15e2ce7e114f46d08df283e2a5b670d7 P adadc55f535f4b0dbe3d04e41251d456: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 06:54:58.888376 22247 mvcc.cc:205] Tried to move back new op lower bound from 7317576494603509760 to 7317576494285987840. Current Snapshot: MvccSnapshot[applied={T|T < 7317576494603509760}]
I20260812 06:54:58.890210 22243 tablet_replica.cc:439] T 15e2ce7e114f46d08df283e2a5b670d7 P cf0e18c976a24d659861decac8e6b157: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "cf0e18c976a24d659861decac8e6b157" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf0e18c976a24d659861decac8e6b157" member_type: VOTER last_known_addr { host: "127.19.52.65" port: 45981 } } peers { permanent_uuid: "adadc55f535f4b0dbe3d04e41251d456" member_type: VOTER last_known_addr { host: "127.19.52.66" port: 36983 } } peers { permanent_uuid: "6a49c31e22d740fab75a5123639a3502" member_type: VOTER last_known_addr { host: "127.19.52.67" port: 44177 } } }
I20260812 06:54:58.890348 22243 tablet_replica.cc:442] T 15e2ce7e114f46d08df283e2a5b670d7 P cf0e18c976a24d659861decac8e6b157: This TxnStatusTablet replica's current role is: LEADER
I20260812 06:54:58.893000 22251 mvcc.cc:205] Tried to move back new op lower bound from 7317576494603509760 to 7317576494285987840. Current Snapshot: MvccSnapshot[applied={T|T < 7317576494603509760}]
I20260812 06:54:58.893338 22225 tablet_replica.cc:439] T 15e2ce7e114f46d08df283e2a5b670d7 P 6a49c31e22d740fab75a5123639a3502: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "cf0e18c976a24d659861decac8e6b157" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf0e18c976a24d659861decac8e6b157" member_type: VOTER last_known_addr { host: "127.19.52.65" port: 45981 } } peers { permanent_uuid: "adadc55f535f4b0dbe3d04e41251d456" member_type: VOTER last_known_addr { host: "127.19.52.66" port: 36983 } } peers { permanent_uuid: "6a49c31e22d740fab75a5123639a3502" member_type: VOTER last_known_addr { host: "127.19.52.67" port: 44177 } } }
I20260812 06:54:58.893483 22225 tablet_replica.cc:442] T 15e2ce7e114f46d08df283e2a5b670d7 P 6a49c31e22d740fab75a5123639a3502: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 06:54:58.899536 22254 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:54:58.894188 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47246 (local address 127.19.52.65:45981)
0812 06:54:58.895623 (+ 1435us) server_negotiation.cc:206] Beginning negotiation
0812 06:54:58.895628 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 06:54:58.895644 (+ 16us) server_negotiation.cc:407] Connection header received
0812 06:54:58.895687 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:54:58.895690 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:54:58.895736 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:54:58.895818 (+ 82us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 06:54:58.896126 (+ 308us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:54:58.896958 (+ 832us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:54:58.898331 (+ 1373us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:54:58.898580 (+ 249us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:54:58.898713 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:54:58.899005 (+ 292us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 06:54:58.899086 (+ 81us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 06:54:58.899183 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0812 06:54:58.899318 (+ 135us) server_negotiation.cc:299] Negotiation successful
0812 06:54:58.899379 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1347}
I20260812 06:54:58.912472 22254 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:54:58.908095 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.52.65:54891 (local address 127.19.52.65:45981)
0812 06:54:58.908220 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0812 06:54:58.908223 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 06:54:58.908459 (+ 236us) server_negotiation.cc:407] Connection header received
0812 06:54:58.908685 (+ 226us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:54:58.908689 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:54:58.908747 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:54:58.908862 (+ 115us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 06:54:58.909152 (+ 290us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:54:58.910014 (+ 862us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:54:58.911110 (+ 1096us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:54:58.911475 (+ 365us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:54:58.911595 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:54:58.911920 (+ 325us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 06:54:58.912010 (+ 90us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 06:54:58.912101 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0812 06:54:58.912247 (+ 146us) server_negotiation.cc:299] Negotiation successful
0812 06:54:58.912304 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260812 06:54:58.912510 22266 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:54:58.907981 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.65:45981 (local address 127.19.52.65:54891)
0812 06:54:58.908370 (+ 389us) negotiation.cc:107] Waiting for socket to connect
0812 06:54:58.908382 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 06:54:58.908524 (+ 142us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:54:58.908930 (+ 406us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:54:58.908932 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:54:58.908940 (+ 8us) client_negotiation.cc:189] Negotiated authn=TOKEN
0812 06:54:58.909030 (+ 90us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:54:58.909035 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:54:58.910154 (+ 1119us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:54:58.910157 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:54:58.910918 (+ 761us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:54:58.910933 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:54:58.911622 (+ 689us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:54:58.911625 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:54:58.911699 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:54:58.911746 (+ 47us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0812 06:54:58.912129 (+ 383us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0812 06:54:58.912136 (+ 7us) client_negotiation.cc:769] Sending connection context
0812 06:54:58.912206 (+ 70us) client_negotiation.cc:240] Negotiation successful
0812 06:54:58.912275 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":315,"thread_start_us":125,"threads_started":1}
I20260812 06:54:58.968833 21632 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46614:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 06:54:58.969058 21632 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46614:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 06:54:58.971989 21632 catalog_manager.cc:5782] T 00000000000000000000000000000000 P edbfe34f29a1474d96de7496f4a45b6f: Sending DeleteTablet for 3 replicas of tablet 6e1ee82e0b0641ca8d086ef3f540c762
I20260812 06:54:58.973002 22117 tablet_service.cc:1514] Processing DeleteTablet for tablet 6e1ee82e0b0641ca8d086ef3f540c762 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 06:54:58 UTC) from {username='kudu', principal='kudu/127.19.52.126@KRBTEST.COM'} at 127.0.0.1:43562
I20260812 06:54:58.973228 21953 tablet_service.cc:1514] Processing DeleteTablet for tablet 6e1ee82e0b0641ca8d086ef3f540c762 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 06:54:58 UTC) from {username='kudu', principal='kudu/127.19.52.126@KRBTEST.COM'} at 127.0.0.1:34660
I20260812 06:54:58.973675 22269 tablet_replica.cc:330] T 6e1ee82e0b0641ca8d086ef3f540c762 P adadc55f535f4b0dbe3d04e41251d456: stopping tablet replica
I20260812 06:54:58.973713 21792 tablet_service.cc:1514] Processing DeleteTablet for tablet 6e1ee82e0b0641ca8d086ef3f540c762 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 06:54:58 UTC) from {username='kudu', principal='kudu/127.19.52.126@KRBTEST.COM'} at 127.0.0.1:47220
I20260812 06:54:58.974056 22269 raft_consensus.cc:2229] T 6e1ee82e0b0641ca8d086ef3f540c762 P adadc55f535f4b0dbe3d04e41251d456 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 06:54:58.974308 22269 raft_consensus.cc:2258] T 6e1ee82e0b0641ca8d086ef3f540c762 P adadc55f535f4b0dbe3d04e41251d456 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 06:54:58.975365 21632 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46614:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 06:54:58.975826 21632 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 06:54:58.975911 22269 ts_tablet_manager.cc:1905] T 6e1ee82e0b0641ca8d086ef3f540c762 P adadc55f535f4b0dbe3d04e41251d456: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 06:54:58.977169 22268 tablet_replica.cc:330] T 6e1ee82e0b0641ca8d086ef3f540c762 P 6a49c31e22d740fab75a5123639a3502: stopping tablet replica
I20260812 06:54:58.977433 22270 tablet_replica.cc:330] T 6e1ee82e0b0641ca8d086ef3f540c762 P cf0e18c976a24d659861decac8e6b157: stopping tablet replica
I20260812 06:54:58.977835 22270 raft_consensus.cc:2229] T 6e1ee82e0b0641ca8d086ef3f540c762 P cf0e18c976a24d659861decac8e6b157 [term 1 LEADER]: Raft consensus shutting down.
I20260812 06:54:58.978220 22268 raft_consensus.cc:2229] T 6e1ee82e0b0641ca8d086ef3f540c762 P 6a49c31e22d740fab75a5123639a3502 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 06:54:58.978519 22268 raft_consensus.cc:2258] T 6e1ee82e0b0641ca8d086ef3f540c762 P 6a49c31e22d740fab75a5123639a3502 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 06:54:58.978965 22270 raft_consensus.cc:2258] T 6e1ee82e0b0641ca8d086ef3f540c762 P cf0e18c976a24d659861decac8e6b157 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 06:54:58.980822 22268 ts_tablet_manager.cc:1905] T 6e1ee82e0b0641ca8d086ef3f540c762 P 6a49c31e22d740fab75a5123639a3502: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 06:54:58.982091 21792 tablet_service.cc:1467] Processing CreateTablet for tablet 7ee4c40804db4d418fb86f33723a46fd (DEFAULT_TABLE table=test-table [id=2f1be7d9de84497f8b60a7b70cc40aad]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 06:54:58.982254 21953 tablet_service.cc:1467] Processing CreateTablet for tablet 7ee4c40804db4d418fb86f33723a46fd (DEFAULT_TABLE table=test-table [id=2f1be7d9de84497f8b60a7b70cc40aad]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 06:54:58.982429 21792 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7ee4c40804db4d418fb86f33723a46fd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 06:54:58.982723 22117 tablet_service.cc:1467] Processing CreateTablet for tablet 7ee4c40804db4d418fb86f33723a46fd (DEFAULT_TABLE table=test-table [id=2f1be7d9de84497f8b60a7b70cc40aad]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 06:54:58.982825 22270 ts_tablet_manager.cc:1905] T 6e1ee82e0b0641ca8d086ef3f540c762 P cf0e18c976a24d659861decac8e6b157: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 06:54:58.983000 22117 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7ee4c40804db4d418fb86f33723a46fd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 06:54:58.986312 21953 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7ee4c40804db4d418fb86f33723a46fd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 06:54:58.989605 22269 ts_tablet_manager.cc:1918] T 6e1ee82e0b0641ca8d086ef3f540c762 P adadc55f535f4b0dbe3d04e41251d456: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260812 06:54:58.989746 22269 log.cc:1196] T 6e1ee82e0b0641ca8d086ef3f540c762 P adadc55f535f4b0dbe3d04e41251d456: Deleting WAL directory at /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786517687384214-19665-0/minicluster-data/ts-1/wal/wals/6e1ee82e0b0641ca8d086ef3f540c762
I20260812 06:54:58.989430 22268 ts_tablet_manager.cc:1918] T 6e1ee82e0b0641ca8d086ef3f540c762 P 6a49c31e22d740fab75a5123639a3502: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260812 06:54:58.990146 22268 log.cc:1196] T 6e1ee82e0b0641ca8d086ef3f540c762 P 6a49c31e22d740fab75a5123639a3502: Deleting WAL directory at /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786517687384214-19665-0/minicluster-data/ts-2/wal/wals/6e1ee82e0b0641ca8d086ef3f540c762
I20260812 06:54:58.990156 22269 ts_tablet_manager.cc:1939] T 6e1ee82e0b0641ca8d086ef3f540c762 P adadc55f535f4b0dbe3d04e41251d456: Deleting consensus metadata
I20260812 06:54:58.990562 22268 ts_tablet_manager.cc:1939] T 6e1ee82e0b0641ca8d086ef3f540c762 P 6a49c31e22d740fab75a5123639a3502: Deleting consensus metadata
I20260812 06:54:58.991581 22270 ts_tablet_manager.cc:1918] T 6e1ee82e0b0641ca8d086ef3f540c762 P cf0e18c976a24d659861decac8e6b157: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260812 06:54:58.991720 22270 log.cc:1196] T 6e1ee82e0b0641ca8d086ef3f540c762 P cf0e18c976a24d659861decac8e6b157: Deleting WAL directory at /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786517687384214-19665-0/minicluster-data/ts-0/wal/wals/6e1ee82e0b0641ca8d086ef3f540c762
I20260812 06:54:58.991660 21620 catalog_manager.cc:4841] TS adadc55f535f4b0dbe3d04e41251d456 (127.19.52.66:36983): tablet 6e1ee82e0b0641ca8d086ef3f540c762 (table test-table [id=72072540752c4142b72c2c5fd18edc84]) successfully deleted
I20260812 06:54:58.991904 21620 catalog_manager.cc:4841] TS 6a49c31e22d740fab75a5123639a3502 (127.19.52.67:44177): tablet 6e1ee82e0b0641ca8d086ef3f540c762 (table test-table [id=72072540752c4142b72c2c5fd18edc84]) successfully deleted
I20260812 06:54:58.992070 22272 tablet_bootstrap.cc:492] T 7ee4c40804db4d418fb86f33723a46fd P 6a49c31e22d740fab75a5123639a3502: Bootstrap starting.
I20260812 06:54:58.992115 22270 ts_tablet_manager.cc:1939] T 6e1ee82e0b0641ca8d086ef3f540c762 P cf0e18c976a24d659861decac8e6b157: Deleting consensus metadata
I20260812 06:54:58.992300 22271 tablet_bootstrap.cc:492] T 7ee4c40804db4d418fb86f33723a46fd P cf0e18c976a24d659861decac8e6b157: Bootstrap starting.
I20260812 06:54:58.993311 22271 tablet_bootstrap.cc:654] T 7ee4c40804db4d418fb86f33723a46fd P cf0e18c976a24d659861decac8e6b157: Neither blocks nor log segments found. Creating new log.
I20260812 06:54:58.993311 22272 tablet_bootstrap.cc:654] T 7ee4c40804db4d418fb86f33723a46fd P 6a49c31e22d740fab75a5123639a3502: Neither blocks nor log segments found. Creating new log.
I20260812 06:54:58.993294 21617 catalog_manager.cc:4841] TS cf0e18c976a24d659861decac8e6b157 (127.19.52.65:45981): tablet 6e1ee82e0b0641ca8d086ef3f540c762 (table test-table [id=72072540752c4142b72c2c5fd18edc84]) successfully deleted
I20260812 06:54:58.994333 22273 tablet_bootstrap.cc:492] T 7ee4c40804db4d418fb86f33723a46fd P adadc55f535f4b0dbe3d04e41251d456: Bootstrap starting.
I20260812 06:54:58.995144 22273 tablet_bootstrap.cc:654] T 7ee4c40804db4d418fb86f33723a46fd P adadc55f535f4b0dbe3d04e41251d456: Neither blocks nor log segments found. Creating new log.
I20260812 06:54:58.996198 22272 tablet_bootstrap.cc:492] T 7ee4c40804db4d418fb86f33723a46fd P 6a49c31e22d740fab75a5123639a3502: No bootstrap required, opened a new log
I20260812 06:54:58.996322 22272 ts_tablet_manager.cc:1397] T 7ee4c40804db4d418fb86f33723a46fd P 6a49c31e22d740fab75a5123639a3502: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260812 06:54:58.996512 22273 tablet_bootstrap.cc:492] T 7ee4c40804db4d418fb86f33723a46fd P adadc55f535f4b0dbe3d04e41251d456: No bootstrap required, opened a new log
I20260812 06:54:58.996654 22273 ts_tablet_manager.cc:1397] T 7ee4c40804db4d418fb86f33723a46fd P adadc55f535f4b0dbe3d04e41251d456: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260812 06:54:58.996932 22272 raft_consensus.cc:348] T 7ee4c40804db4d418fb86f33723a46fd P 6a49c31e22d740fab75a5123639a3502 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf0e18c976a24d659861decac8e6b157" member_type: VOTER last_known_addr { host: "127.19.52.65" port: 45981 } } peers { permanent_uuid: "adadc55f535f4b0dbe3d04e41251d456" member_type: VOTER last_known_addr { host: "127.19.52.66" port: 36983 } } peers { permanent_uuid: "6a49c31e22d740fab75a5123639a3502" member_type: VOTER last_known_addr { host: "127.19.52.67" port: 44177 } }
I20260812 06:54:58.997069 22272 raft_consensus.cc:374] T 7ee4c40804db4d418fb86f33723a46fd P 6a49c31e22d740fab75a5123639a3502 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 06:54:58.997114 22272 raft_consensus.cc:729] T 7ee4c40804db4d418fb86f33723a46fd P 6a49c31e22d740fab75a5123639a3502 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6a49c31e22d740fab75a5123639a3502, State: Initialized, Role: FOLLOWER
I20260812 06:54:58.997139 22273 raft_consensus.cc:348] T 7ee4c40804db4d418fb86f33723a46fd P adadc55f535f4b0dbe3d04e41251d456 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf0e18c976a24d659861decac8e6b157" member_type: VOTER last_known_addr { host: "127.19.52.65" port: 45981 } } peers { permanent_uuid: "adadc55f535f4b0dbe3d04e41251d456" member_type: VOTER last_known_addr { host: "127.19.52.66" port: 36983 } } peers { permanent_uuid: "6a49c31e22d740fab75a5123639a3502" member_type: VOTER last_known_addr { host: "127.19.52.67" port: 44177 } }
I20260812 06:54:58.997265 22273 raft_consensus.cc:374] T 7ee4c40804db4d418fb86f33723a46fd P adadc55f535f4b0dbe3d04e41251d456 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 06:54:58.997298 22273 raft_consensus.cc:729] T 7ee4c40804db4d418fb86f33723a46fd P adadc55f535f4b0dbe3d04e41251d456 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: adadc55f535f4b0dbe3d04e41251d456, State: Initialized, Role: FOLLOWER
I20260812 06:54:58.997303 22272 consensus_queue.cc:260] T 7ee4c40804db4d418fb86f33723a46fd P 6a49c31e22d740fab75a5123639a3502 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf0e18c976a24d659861decac8e6b157" member_type: VOTER last_known_addr { host: "127.19.52.65" port: 45981 } } peers { permanent_uuid: "adadc55f535f4b0dbe3d04e41251d456" member_type: VOTER last_known_addr { host: "127.19.52.66" port: 36983 } } peers { permanent_uuid: "6a49c31e22d740fab75a5123639a3502" member_type: VOTER last_known_addr { host: "127.19.52.67" port: 44177 } }
I20260812 06:54:58.997427 22273 consensus_queue.cc:260] T 7ee4c40804db4d418fb86f33723a46fd P adadc55f535f4b0dbe3d04e41251d456 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf0e18c976a24d659861decac8e6b157" member_type: VOTER last_known_addr { host: "127.19.52.65" port: 45981 } } peers { permanent_uuid: "adadc55f535f4b0dbe3d04e41251d456" member_type: VOTER last_known_addr { host: "127.19.52.66" port: 36983 } } peers { permanent_uuid: "6a49c31e22d740fab75a5123639a3502" member_type: VOTER last_known_addr { host: "127.19.52.67" port: 44177 } }
I20260812 06:54:58.997720 22273 ts_tablet_manager.cc:1428] T 7ee4c40804db4d418fb86f33723a46fd P adadc55f535f4b0dbe3d04e41251d456: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260812 06:54:58.998351 22272 ts_tablet_manager.cc:1428] T 7ee4c40804db4d418fb86f33723a46fd P 6a49c31e22d740fab75a5123639a3502: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260812 06:54:58.999328 22271 tablet_bootstrap.cc:492] T 7ee4c40804db4d418fb86f33723a46fd P cf0e18c976a24d659861decac8e6b157: No bootstrap required, opened a new log
I20260812 06:54:58.999451 22271 ts_tablet_manager.cc:1397] T 7ee4c40804db4d418fb86f33723a46fd P cf0e18c976a24d659861decac8e6b157: Time spent bootstrapping tablet: real 0.007s user 0.002s sys 0.000s
I20260812 06:54:58.999954 22271 raft_consensus.cc:348] T 7ee4c40804db4d418fb86f33723a46fd P cf0e18c976a24d659861decac8e6b157 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf0e18c976a24d659861decac8e6b157" member_type: VOTER last_known_addr { host: "127.19.52.65" port: 45981 } } peers { permanent_uuid: "adadc55f535f4b0dbe3d04e41251d456" member_type: VOTER last_known_addr { host: "127.19.52.66" port: 36983 } } peers { permanent_uuid: "6a49c31e22d740fab75a5123639a3502" member_type: VOTER last_known_addr { host: "127.19.52.67" port: 44177 } }
I20260812 06:54:59.000109 22271 raft_consensus.cc:374] T 7ee4c40804db4d418fb86f33723a46fd P cf0e18c976a24d659861decac8e6b157 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 06:54:59.000165 22271 raft_consensus.cc:729] T 7ee4c40804db4d418fb86f33723a46fd P cf0e18c976a24d659861decac8e6b157 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cf0e18c976a24d659861decac8e6b157, State: Initialized, Role: FOLLOWER
I20260812 06:54:59.000301 22271 consensus_queue.cc:260] T 7ee4c40804db4d418fb86f33723a46fd P cf0e18c976a24d659861decac8e6b157 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf0e18c976a24d659861decac8e6b157" member_type: VOTER last_known_addr { host: "127.19.52.65" port: 45981 } } peers { permanent_uuid: "adadc55f535f4b0dbe3d04e41251d456" member_type: VOTER last_known_addr { host: "127.19.52.66" port: 36983 } } peers { permanent_uuid: "6a49c31e22d740fab75a5123639a3502" member_type: VOTER last_known_addr { host: "127.19.52.67" port: 44177 } }
I20260812 06:54:59.000654 22271 ts_tablet_manager.cc:1428] T 7ee4c40804db4d418fb86f33723a46fd P cf0e18c976a24d659861decac8e6b157: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260812 06:54:59.160624 22243 raft_consensus.cc:482] T 7ee4c40804db4d418fb86f33723a46fd P cf0e18c976a24d659861decac8e6b157 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 06:54:59.160785 22243 raft_consensus.cc:504] T 7ee4c40804db4d418fb86f33723a46fd P cf0e18c976a24d659861decac8e6b157 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf0e18c976a24d659861decac8e6b157" member_type: VOTER last_known_addr { host: "127.19.52.65" port: 45981 } } peers { permanent_uuid: "adadc55f535f4b0dbe3d04e41251d456" member_type: VOTER last_known_addr { host: "127.19.52.66" port: 36983 } } peers { permanent_uuid: "6a49c31e22d740fab75a5123639a3502" member_type: VOTER last_known_addr { host: "127.19.52.67" port: 44177 } }
I20260812 06:54:59.161338 22243 leader_election.cc:288] T 7ee4c40804db4d418fb86f33723a46fd P cf0e18c976a24d659861decac8e6b157 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers adadc55f535f4b0dbe3d04e41251d456 (127.19.52.66:36983), 6a49c31e22d740fab75a5123639a3502 (127.19.52.67:44177)
I20260812 06:54:59.161995 21981 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7ee4c40804db4d418fb86f33723a46fd" candidate_uuid: "cf0e18c976a24d659861decac8e6b157" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "adadc55f535f4b0dbe3d04e41251d456" is_pre_election: true
I20260812 06:54:59.162050 22140 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7ee4c40804db4d418fb86f33723a46fd" candidate_uuid: "cf0e18c976a24d659861decac8e6b157" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6a49c31e22d740fab75a5123639a3502" is_pre_election: true
I20260812 06:54:59.162225 21981 raft_consensus.cc:2454] T 7ee4c40804db4d418fb86f33723a46fd P adadc55f535f4b0dbe3d04e41251d456 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cf0e18c976a24d659861decac8e6b157 in term 0.
I20260812 06:54:59.162285 22140 raft_consensus.cc:2454] T 7ee4c40804db4d418fb86f33723a46fd P 6a49c31e22d740fab75a5123639a3502 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cf0e18c976a24d659861decac8e6b157 in term 0.
I20260812 06:54:59.162562 21741 leader_election.cc:302] T 7ee4c40804db4d418fb86f33723a46fd P cf0e18c976a24d659861decac8e6b157 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: adadc55f535f4b0dbe3d04e41251d456, cf0e18c976a24d659861decac8e6b157; no voters:
I20260812 06:54:59.162961 22243 raft_consensus.cc:2781] T 7ee4c40804db4d418fb86f33723a46fd P cf0e18c976a24d659861decac8e6b157 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 06:54:59.163040 22243 raft_consensus.cc:482] T 7ee4c40804db4d418fb86f33723a46fd P cf0e18c976a24d659861decac8e6b157 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 06:54:59.163100 22243 raft_consensus.cc:3037] T 7ee4c40804db4d418fb86f33723a46fd P cf0e18c976a24d659861decac8e6b157 [term 0 FOLLOWER]: Advancing to term 1
I20260812 06:54:59.164356 22243 raft_consensus.cc:504] T 7ee4c40804db4d418fb86f33723a46fd P cf0e18c976a24d659861decac8e6b157 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf0e18c976a24d659861decac8e6b157" member_type: VOTER last_known_addr { host: "127.19.52.65" port: 45981 } } peers { permanent_uuid: "adadc55f535f4b0dbe3d04e41251d456" member_type: VOTER last_known_addr { host: "127.19.52.66" port: 36983 } } peers { permanent_uuid: "6a49c31e22d740fab75a5123639a3502" member_type: VOTER last_known_addr { host: "127.19.52.67" port: 44177 } }
I20260812 06:54:59.164794 22243 leader_election.cc:288] T 7ee4c40804db4d418fb86f33723a46fd P cf0e18c976a24d659861decac8e6b157 [CANDIDATE]: Term 1 election: Requested vote from peers adadc55f535f4b0dbe3d04e41251d456 (127.19.52.66:36983), 6a49c31e22d740fab75a5123639a3502 (127.19.52.67:44177)
I20260812 06:54:59.165225 22140 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7ee4c40804db4d418fb86f33723a46fd" candidate_uuid: "cf0e18c976a24d659861decac8e6b157" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6a49c31e22d740fab75a5123639a3502"
I20260812 06:54:59.165225 21981 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7ee4c40804db4d418fb86f33723a46fd" candidate_uuid: "cf0e18c976a24d659861decac8e6b157" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "adadc55f535f4b0dbe3d04e41251d456"
I20260812 06:54:59.165342 22140 raft_consensus.cc:3037] T 7ee4c40804db4d418fb86f33723a46fd P 6a49c31e22d740fab75a5123639a3502 [term 0 FOLLOWER]: Advancing to term 1
I20260812 06:54:59.165375 21981 raft_consensus.cc:3037] T 7ee4c40804db4d418fb86f33723a46fd P adadc55f535f4b0dbe3d04e41251d456 [term 0 FOLLOWER]: Advancing to term 1
I20260812 06:54:59.166391 22140 raft_consensus.cc:2454] T 7ee4c40804db4d418fb86f33723a46fd P 6a49c31e22d740fab75a5123639a3502 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cf0e18c976a24d659861decac8e6b157 in term 1.
I20260812 06:54:59.166437 21981 raft_consensus.cc:2454] T 7ee4c40804db4d418fb86f33723a46fd P adadc55f535f4b0dbe3d04e41251d456 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cf0e18c976a24d659861decac8e6b157 in term 1.
I20260812 06:54:59.166759 21741 leader_election.cc:302] T 7ee4c40804db4d418fb86f33723a46fd P cf0e18c976a24d659861decac8e6b157 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a49c31e22d740fab75a5123639a3502, cf0e18c976a24d659861decac8e6b157; no voters:
I20260812 06:54:59.166939 22243 raft_consensus.cc:2781] T 7ee4c40804db4d418fb86f33723a46fd P cf0e18c976a24d659861decac8e6b157 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 06:54:59.167090 22243 raft_consensus.cc:686] T 7ee4c40804db4d418fb86f33723a46fd P cf0e18c976a24d659861decac8e6b157 [term 1 LEADER]: Becoming Leader. State: Replica: cf0e18c976a24d659861decac8e6b157, State: Running, Role: LEADER
I20260812 06:54:59.167265 22243 consensus_queue.cc:237] T 7ee4c40804db4d418fb86f33723a46fd P cf0e18c976a24d659861decac8e6b157 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf0e18c976a24d659861decac8e6b157" member_type: VOTER last_known_addr { host: "127.19.52.65" port: 45981 } } peers { permanent_uuid: "adadc55f535f4b0dbe3d04e41251d456" member_type: VOTER last_known_addr { host: "127.19.52.66" port: 36983 } } peers { permanent_uuid: "6a49c31e22d740fab75a5123639a3502" member_type: VOTER last_known_addr { host: "127.19.52.67" port: 44177 } }
I20260812 06:54:59.169294 21633 catalog_manager.cc:5495] T 7ee4c40804db4d418fb86f33723a46fd P cf0e18c976a24d659861decac8e6b157 reported cstate change: term changed from 0 to 1, leader changed from <none> to cf0e18c976a24d659861decac8e6b157 (127.19.52.65). New cstate: current_term: 1 leader_uuid: "cf0e18c976a24d659861decac8e6b157" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf0e18c976a24d659861decac8e6b157" member_type: VOTER last_known_addr { host: "127.19.52.65" port: 45981 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "adadc55f535f4b0dbe3d04e41251d456" member_type: VOTER last_known_addr { host: "127.19.52.66" port: 36983 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6a49c31e22d740fab75a5123639a3502" member_type: VOTER last_known_addr { host: "127.19.52.67" port: 44177 } health_report { overall_health: UNKNOWN } } }
I20260812 06:54:59.195355 21980 raft_consensus.cc:1261] T 7ee4c40804db4d418fb86f33723a46fd P adadc55f535f4b0dbe3d04e41251d456 [term 1 FOLLOWER]: Refusing update from remote peer cf0e18c976a24d659861decac8e6b157: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 06:54:59.195901 22140 raft_consensus.cc:1261] T 7ee4c40804db4d418fb86f33723a46fd P 6a49c31e22d740fab75a5123639a3502 [term 1 FOLLOWER]: Refusing update from remote peer cf0e18c976a24d659861decac8e6b157: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 06:54:59.195919 22224 consensus_queue.cc:1035] T 7ee4c40804db4d418fb86f33723a46fd P cf0e18c976a24d659861decac8e6b157 [LEADER]: Connected to new peer: Peer: permanent_uuid: "adadc55f535f4b0dbe3d04e41251d456" member_type: VOTER last_known_addr { host: "127.19.52.66" port: 36983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 06:54:59.196241 22224 consensus_queue.cc:1035] T 7ee4c40804db4d418fb86f33723a46fd P cf0e18c976a24d659861decac8e6b157 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a49c31e22d740fab75a5123639a3502" member_type: VOTER last_known_addr { host: "127.19.52.67" port: 44177 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260812 06:54:59.212163 22244 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260812 06:55:00.778388 21633 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46614:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 06:55:00.778584 21633 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46614:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 06:55:00.781287 21633 catalog_manager.cc:5782] T 00000000000000000000000000000000 P edbfe34f29a1474d96de7496f4a45b6f: Sending DeleteTablet for 3 replicas of tablet 7ee4c40804db4d418fb86f33723a46fd
I20260812 06:55:00.781839 21792 tablet_service.cc:1514] Processing DeleteTablet for tablet 7ee4c40804db4d418fb86f33723a46fd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 06:55:00 UTC) from {username='kudu', principal='kudu/127.19.52.126@KRBTEST.COM'} at 127.0.0.1:47220
I20260812 06:55:00.781982 21953 tablet_service.cc:1514] Processing DeleteTablet for tablet 7ee4c40804db4d418fb86f33723a46fd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 06:55:00 UTC) from {username='kudu', principal='kudu/127.19.52.126@KRBTEST.COM'} at 127.0.0.1:34660
I20260812 06:55:00.782100 22117 tablet_service.cc:1514] Processing DeleteTablet for tablet 7ee4c40804db4d418fb86f33723a46fd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 06:55:00 UTC) from {username='kudu', principal='kudu/127.19.52.126@KRBTEST.COM'} at 127.0.0.1:43562
I20260812 06:55:00.782382 22305 tablet_replica.cc:330] T 7ee4c40804db4d418fb86f33723a46fd P cf0e18c976a24d659861decac8e6b157: stopping tablet replica
I20260812 06:55:00.782580 22305 raft_consensus.cc:2229] T 7ee4c40804db4d418fb86f33723a46fd P cf0e18c976a24d659861decac8e6b157 [term 1 LEADER]: Raft consensus shutting down.
I20260812 06:55:00.782624 22306 tablet_replica.cc:330] T 7ee4c40804db4d418fb86f33723a46fd P adadc55f535f4b0dbe3d04e41251d456: stopping tablet replica
I20260812 06:55:00.782832 22305 raft_consensus.cc:2258] T 7ee4c40804db4d418fb86f33723a46fd P cf0e18c976a24d659861decac8e6b157 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 06:55:00.782871 22306 raft_consensus.cc:2229] T 7ee4c40804db4d418fb86f33723a46fd P adadc55f535f4b0dbe3d04e41251d456 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 06:55:00.783035 22306 raft_consensus.cc:2258] T 7ee4c40804db4d418fb86f33723a46fd P adadc55f535f4b0dbe3d04e41251d456 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 06:55:00.783552 22305 ts_tablet_manager.cc:1905] T 7ee4c40804db4d418fb86f33723a46fd P cf0e18c976a24d659861decac8e6b157: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 06:55:00.785198 22306 ts_tablet_manager.cc:1905] T 7ee4c40804db4d418fb86f33723a46fd P adadc55f535f4b0dbe3d04e41251d456: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 06:55:00.785785 22307 tablet_replica.cc:330] T 7ee4c40804db4d418fb86f33723a46fd P 6a49c31e22d740fab75a5123639a3502: stopping tablet replica
I20260812 06:55:00.785925 22307 raft_consensus.cc:2229] T 7ee4c40804db4d418fb86f33723a46fd P 6a49c31e22d740fab75a5123639a3502 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 06:55:00.786044 22307 raft_consensus.cc:2258] T 7ee4c40804db4d418fb86f33723a46fd P 6a49c31e22d740fab75a5123639a3502 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 06:55:00.786798 22305 ts_tablet_manager.cc:1918] T 7ee4c40804db4d418fb86f33723a46fd P cf0e18c976a24d659861decac8e6b157: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260812 06:55:00.786899 22305 log.cc:1196] T 7ee4c40804db4d418fb86f33723a46fd P cf0e18c976a24d659861decac8e6b157: Deleting WAL directory at /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786517687384214-19665-0/minicluster-data/ts-0/wal/wals/7ee4c40804db4d418fb86f33723a46fd
I20260812 06:55:00.787199 22305 ts_tablet_manager.cc:1939] T 7ee4c40804db4d418fb86f33723a46fd P cf0e18c976a24d659861decac8e6b157: Deleting consensus metadata
I20260812 06:55:00.787787 22307 ts_tablet_manager.cc:1905] T 7ee4c40804db4d418fb86f33723a46fd P 6a49c31e22d740fab75a5123639a3502: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 06:55:00.788136 21617 catalog_manager.cc:4841] TS cf0e18c976a24d659861decac8e6b157 (127.19.52.65:45981): tablet 7ee4c40804db4d418fb86f33723a46fd (table test-table [id=2f1be7d9de84497f8b60a7b70cc40aad]) successfully deleted
I20260812 06:55:00.788875 22306 ts_tablet_manager.cc:1918] T 7ee4c40804db4d418fb86f33723a46fd P adadc55f535f4b0dbe3d04e41251d456: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260812 06:55:00.788970 22306 log.cc:1196] T 7ee4c40804db4d418fb86f33723a46fd P adadc55f535f4b0dbe3d04e41251d456: Deleting WAL directory at /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786517687384214-19665-0/minicluster-data/ts-1/wal/wals/7ee4c40804db4d418fb86f33723a46fd
I20260812 06:55:00.789310 22306 ts_tablet_manager.cc:1939] T 7ee4c40804db4d418fb86f33723a46fd P adadc55f535f4b0dbe3d04e41251d456: Deleting consensus metadata
I20260812 06:55:00.790139 21620 catalog_manager.cc:4841] TS adadc55f535f4b0dbe3d04e41251d456 (127.19.52.66:36983): tablet 7ee4c40804db4d418fb86f33723a46fd (table test-table [id=2f1be7d9de84497f8b60a7b70cc40aad]) successfully deleted
I20260812 06:55:00.790535 22307 ts_tablet_manager.cc:1918] T 7ee4c40804db4d418fb86f33723a46fd P 6a49c31e22d740fab75a5123639a3502: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260812 06:55:00.790630 22307 log.cc:1196] T 7ee4c40804db4d418fb86f33723a46fd P 6a49c31e22d740fab75a5123639a3502: Deleting WAL directory at /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786517687384214-19665-0/minicluster-data/ts-2/wal/wals/7ee4c40804db4d418fb86f33723a46fd
I20260812 06:55:00.790910 22307 ts_tablet_manager.cc:1939] T 7ee4c40804db4d418fb86f33723a46fd P 6a49c31e22d740fab75a5123639a3502: Deleting consensus metadata
I20260812 06:55:00.791776 21620 catalog_manager.cc:4841] TS 6a49c31e22d740fab75a5123639a3502 (127.19.52.67:44177): tablet 7ee4c40804db4d418fb86f33723a46fd (table test-table [id=2f1be7d9de84497f8b60a7b70cc40aad]) successfully deleted
Aug 12 06:55:00 dist-test-slave-65mx krb5kdc[21585](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517700, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 12 06:55:00 dist-test-slave-65mx krb5kdc[21585](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517700, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.19.52.126@KRBTEST.COM
I20260812 06:55:00.814222 22316 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:00.804292 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46644 (local address 127.19.52.126:43141)
0812 06:55:00.804675 (+ 383us) server_negotiation.cc:206] Beginning negotiation
0812 06:55:00.804688 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0812 06:55:00.804713 (+ 25us) server_negotiation.cc:407] Connection header received
0812 06:55:00.804914 (+ 201us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:55:00.804917 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:55:00.804974 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:55:00.805088 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:55:00.805837 (+ 749us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:00.806396 (+ 559us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:00.807279 (+ 883us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:00.807527 (+ 248us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:00.807641 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:55:00.810491 (+ 2850us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:55:00.810494 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:55:00.810497 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:55:00.810527 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:55:00.812226 (+ 1699us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:55:00.812798 (+ 572us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:55:00.812801 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:55:00.812802 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:55:00.812910 (+ 108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:55:00.813311 (+ 401us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:55:00.813314 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:55:00.813316 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:55:00.813561 (+ 245us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:55:00.813679 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0812 06:55:00.813939 (+ 260us) server_negotiation.cc:299] Negotiation successful
0812 06:55:00.814070 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":237,"thread_start_us":99,"threads_started":1}
I20260812 06:55:00.817468 21633 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:46644:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 06:55:00.818007 21633 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 06:55:00.823314 21792 tablet_service.cc:1467] Processing CreateTablet for tablet d9eb216c97d441278a7446d729ed378b (DEFAULT_TABLE table=test-table [id=65a35ef676064a158d48153bfbfa55fb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 06:55:00.823377 22117 tablet_service.cc:1467] Processing CreateTablet for tablet d9eb216c97d441278a7446d729ed378b (DEFAULT_TABLE table=test-table [id=65a35ef676064a158d48153bfbfa55fb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 06:55:00.823756 22117 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d9eb216c97d441278a7446d729ed378b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 06:55:00.823781 21792 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d9eb216c97d441278a7446d729ed378b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 06:55:00.826414 22317 tablet_bootstrap.cc:492] T d9eb216c97d441278a7446d729ed378b P 6a49c31e22d740fab75a5123639a3502: Bootstrap starting.
I20260812 06:55:00.826695 22318 tablet_bootstrap.cc:492] T d9eb216c97d441278a7446d729ed378b P cf0e18c976a24d659861decac8e6b157: Bootstrap starting.
I20260812 06:55:00.826773 21953 tablet_service.cc:1467] Processing CreateTablet for tablet d9eb216c97d441278a7446d729ed378b (DEFAULT_TABLE table=test-table [id=65a35ef676064a158d48153bfbfa55fb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 06:55:00.827082 21953 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d9eb216c97d441278a7446d729ed378b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 06:55:00.827652 22318 tablet_bootstrap.cc:654] T d9eb216c97d441278a7446d729ed378b P cf0e18c976a24d659861decac8e6b157: Neither blocks nor log segments found. Creating new log.
I20260812 06:55:00.827867 22317 tablet_bootstrap.cc:654] T d9eb216c97d441278a7446d729ed378b P 6a49c31e22d740fab75a5123639a3502: Neither blocks nor log segments found. Creating new log.
I20260812 06:55:00.828948 22318 tablet_bootstrap.cc:492] T d9eb216c97d441278a7446d729ed378b P cf0e18c976a24d659861decac8e6b157: No bootstrap required, opened a new log
I20260812 06:55:00.829058 22318 ts_tablet_manager.cc:1397] T d9eb216c97d441278a7446d729ed378b P cf0e18c976a24d659861decac8e6b157: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260812 06:55:00.829501 22318 raft_consensus.cc:348] T d9eb216c97d441278a7446d729ed378b P cf0e18c976a24d659861decac8e6b157 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf0e18c976a24d659861decac8e6b157" member_type: VOTER last_known_addr { host: "127.19.52.65" port: 45981 } } peers { permanent_uuid: "adadc55f535f4b0dbe3d04e41251d456" member_type: VOTER last_known_addr { host: "127.19.52.66" port: 36983 } } peers { permanent_uuid: "6a49c31e22d740fab75a5123639a3502" member_type: VOTER last_known_addr { host: "127.19.52.67" port: 44177 } }
I20260812 06:55:00.829627 22318 raft_consensus.cc:374] T d9eb216c97d441278a7446d729ed378b P cf0e18c976a24d659861decac8e6b157 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 06:55:00.829672 22318 raft_consensus.cc:729] T d9eb216c97d441278a7446d729ed378b P cf0e18c976a24d659861decac8e6b157 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cf0e18c976a24d659861decac8e6b157, State: Initialized, Role: FOLLOWER
I20260812 06:55:00.829823 22318 consensus_queue.cc:260] T d9eb216c97d441278a7446d729ed378b P cf0e18c976a24d659861decac8e6b157 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf0e18c976a24d659861decac8e6b157" member_type: VOTER last_known_addr { host: "127.19.52.65" port: 45981 } } peers { permanent_uuid: "adadc55f535f4b0dbe3d04e41251d456" member_type: VOTER last_known_addr { host: "127.19.52.66" port: 36983 } } peers { permanent_uuid: "6a49c31e22d740fab75a5123639a3502" member_type: VOTER last_known_addr { host: "127.19.52.67" port: 44177 } }
I20260812 06:55:00.830152 22318 ts_tablet_manager.cc:1428] T d9eb216c97d441278a7446d729ed378b P cf0e18c976a24d659861decac8e6b157: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260812 06:55:00.830255 22321 tablet_bootstrap.cc:492] T d9eb216c97d441278a7446d729ed378b P adadc55f535f4b0dbe3d04e41251d456: Bootstrap starting.
I20260812 06:55:00.831045 22317 tablet_bootstrap.cc:492] T d9eb216c97d441278a7446d729ed378b P 6a49c31e22d740fab75a5123639a3502: No bootstrap required, opened a new log
I20260812 06:55:00.831074 22321 tablet_bootstrap.cc:654] T d9eb216c97d441278a7446d729ed378b P adadc55f535f4b0dbe3d04e41251d456: Neither blocks nor log segments found. Creating new log.
I20260812 06:55:00.831164 22317 ts_tablet_manager.cc:1397] T d9eb216c97d441278a7446d729ed378b P 6a49c31e22d740fab75a5123639a3502: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20260812 06:55:00.831744 22317 raft_consensus.cc:348] T d9eb216c97d441278a7446d729ed378b P 6a49c31e22d740fab75a5123639a3502 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf0e18c976a24d659861decac8e6b157" member_type: VOTER last_known_addr { host: "127.19.52.65" port: 45981 } } peers { permanent_uuid: "adadc55f535f4b0dbe3d04e41251d456" member_type: VOTER last_known_addr { host: "127.19.52.66" port: 36983 } } peers { permanent_uuid: "6a49c31e22d740fab75a5123639a3502" member_type: VOTER last_known_addr { host: "127.19.52.67" port: 44177 } }
I20260812 06:55:00.831881 22317 raft_consensus.cc:374] T d9eb216c97d441278a7446d729ed378b P 6a49c31e22d740fab75a5123639a3502 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 06:55:00.831941 22317 raft_consensus.cc:729] T d9eb216c97d441278a7446d729ed378b P 6a49c31e22d740fab75a5123639a3502 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6a49c31e22d740fab75a5123639a3502, State: Initialized, Role: FOLLOWER
I20260812 06:55:00.832083 22317 consensus_queue.cc:260] T d9eb216c97d441278a7446d729ed378b P 6a49c31e22d740fab75a5123639a3502 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf0e18c976a24d659861decac8e6b157" member_type: VOTER last_known_addr { host: "127.19.52.65" port: 45981 } } peers { permanent_uuid: "adadc55f535f4b0dbe3d04e41251d456" member_type: VOTER last_known_addr { host: "127.19.52.66" port: 36983 } } peers { permanent_uuid: "6a49c31e22d740fab75a5123639a3502" member_type: VOTER last_known_addr { host: "127.19.52.67" port: 44177 } }
I20260812 06:55:00.832577 22317 ts_tablet_manager.cc:1428] T d9eb216c97d441278a7446d729ed378b P 6a49c31e22d740fab75a5123639a3502: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260812 06:55:00.833294 22321 tablet_bootstrap.cc:492] T d9eb216c97d441278a7446d729ed378b P adadc55f535f4b0dbe3d04e41251d456: No bootstrap required, opened a new log
I20260812 06:55:00.833396 22321 ts_tablet_manager.cc:1397] T d9eb216c97d441278a7446d729ed378b P adadc55f535f4b0dbe3d04e41251d456: Time spent bootstrapping tablet: real 0.003s user 0.003s sys 0.000s
I20260812 06:55:00.833916 22321 raft_consensus.cc:348] T d9eb216c97d441278a7446d729ed378b P adadc55f535f4b0dbe3d04e41251d456 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf0e18c976a24d659861decac8e6b157" member_type: VOTER last_known_addr { host: "127.19.52.65" port: 45981 } } peers { permanent_uuid: "adadc55f535f4b0dbe3d04e41251d456" member_type: VOTER last_known_addr { host: "127.19.52.66" port: 36983 } } peers { permanent_uuid: "6a49c31e22d740fab75a5123639a3502" member_type: VOTER last_known_addr { host: "127.19.52.67" port: 44177 } }
I20260812 06:55:00.834013 22321 raft_consensus.cc:374] T d9eb216c97d441278a7446d729ed378b P adadc55f535f4b0dbe3d04e41251d456 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 06:55:00.834035 22321 raft_consensus.cc:729] T d9eb216c97d441278a7446d729ed378b P adadc55f535f4b0dbe3d04e41251d456 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: adadc55f535f4b0dbe3d04e41251d456, State: Initialized, Role: FOLLOWER
I20260812 06:55:00.834177 22321 consensus_queue.cc:260] T d9eb216c97d441278a7446d729ed378b P adadc55f535f4b0dbe3d04e41251d456 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf0e18c976a24d659861decac8e6b157" member_type: VOTER last_known_addr { host: "127.19.52.65" port: 45981 } } peers { permanent_uuid: "adadc55f535f4b0dbe3d04e41251d456" member_type: VOTER last_known_addr { host: "127.19.52.66" port: 36983 } } peers { permanent_uuid: "6a49c31e22d740fab75a5123639a3502" member_type: VOTER last_known_addr { host: "127.19.52.67" port: 44177 } }
I20260812 06:55:00.834596 22321 ts_tablet_manager.cc:1428] T d9eb216c97d441278a7446d729ed378b P adadc55f535f4b0dbe3d04e41251d456: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260812 06:55:00.868752 22325 raft_consensus.cc:482] T d9eb216c97d441278a7446d729ed378b P adadc55f535f4b0dbe3d04e41251d456 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 06:55:00.868911 22325 raft_consensus.cc:504] T d9eb216c97d441278a7446d729ed378b P adadc55f535f4b0dbe3d04e41251d456 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf0e18c976a24d659861decac8e6b157" member_type: VOTER last_known_addr { host: "127.19.52.65" port: 45981 } } peers { permanent_uuid: "adadc55f535f4b0dbe3d04e41251d456" member_type: VOTER last_known_addr { host: "127.19.52.66" port: 36983 } } peers { permanent_uuid: "6a49c31e22d740fab75a5123639a3502" member_type: VOTER last_known_addr { host: "127.19.52.67" port: 44177 } }
I20260812 06:55:00.869987 22325 leader_election.cc:288] T d9eb216c97d441278a7446d729ed378b P adadc55f535f4b0dbe3d04e41251d456 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cf0e18c976a24d659861decac8e6b157 (127.19.52.65:45981), 6a49c31e22d740fab75a5123639a3502 (127.19.52.67:44177)
I20260812 06:55:00.874219 22329 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:00.870487 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.52.66:37779 (local address 127.19.52.65:45981)
0812 06:55:00.870776 (+ 289us) server_negotiation.cc:206] Beginning negotiation
0812 06:55:00.870787 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0812 06:55:00.870804 (+ 17us) server_negotiation.cc:407] Connection header received
0812 06:55:00.870975 (+ 171us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:55:00.870978 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:55:00.871033 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:55:00.871129 (+ 96us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 06:55:00.871404 (+ 275us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:00.871940 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:00.873137 (+ 1197us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:00.873594 (+ 457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:00.873728 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:55:00.873790 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0812 06:55:00.873984 (+ 194us) server_negotiation.cc:299] Negotiation successful
0812 06:55:00.874068 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":199,"thread_start_us":118,"threads_started":1}
I20260812 06:55:00.874228 22326 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:00.870258 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.65:45981 (local address 127.19.52.66:37779)
0812 06:55:00.870704 (+ 446us) negotiation.cc:107] Waiting for socket to connect
0812 06:55:00.870721 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0812 06:55:00.870842 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:55:00.871139 (+ 297us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:55:00.871142 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:55:00.871158 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 06:55:00.871288 (+ 130us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:55:00.871294 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:55:00.872074 (+ 780us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:55:00.872079 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:55:00.873005 (+ 926us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:55:00.873020 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:55:00.873725 (+ 705us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:55:00.873729 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:55:00.873844 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:55:00.873875 (+ 31us) client_negotiation.cc:769] Sending connection context
0812 06:55:00.873965 (+ 90us) client_negotiation.cc:240] Negotiation successful
0812 06:55:00.874063 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":363,"mutex_wait_us":2,"thread_start_us":163,"threads_started":1}
I20260812 06:55:00.874243 22328 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:00.870367 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.52.66:52743 (local address 127.19.52.67:44177)
0812 06:55:00.871011 (+ 644us) server_negotiation.cc:206] Beginning negotiation
0812 06:55:00.871017 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 06:55:00.871028 (+ 11us) server_negotiation.cc:407] Connection header received
0812 06:55:00.871070 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:55:00.871072 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:55:00.871131 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:55:00.871231 (+ 100us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 06:55:00.871506 (+ 275us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:00.872063 (+ 557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:00.873164 (+ 1101us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:00.873597 (+ 433us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:00.873747 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:55:00.873806 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0812 06:55:00.873999 (+ 193us) server_negotiation.cc:299] Negotiation successful
0812 06:55:00.874081 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":579,"thread_start_us":116,"threads_started":1}
I20260812 06:55:00.874246 22327 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:00.870258 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.67:44177 (local address 127.19.52.66:52743)
0812 06:55:00.870768 (+ 510us) negotiation.cc:107] Waiting for socket to connect
0812 06:55:00.870781 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 06:55:00.870864 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:55:00.871246 (+ 382us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:55:00.871249 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:55:00.871265 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 06:55:00.871383 (+ 118us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:55:00.871388 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:55:00.872184 (+ 796us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:55:00.872189 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:55:00.873038 (+ 849us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:55:00.873046 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:55:00.873746 (+ 700us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:55:00.873750 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:55:00.873876 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:55:00.873893 (+ 17us) client_negotiation.cc:769] Sending connection context
0812 06:55:00.873981 (+ 88us) client_negotiation.cc:240] Negotiation successful
0812 06:55:00.874079 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":450,"mutex_wait_us":85,"thread_start_us":143,"threads_started":1}
I20260812 06:55:00.874820 22140 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d9eb216c97d441278a7446d729ed378b" candidate_uuid: "adadc55f535f4b0dbe3d04e41251d456" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6a49c31e22d740fab75a5123639a3502" is_pre_election: true
I20260812 06:55:00.875013 22140 raft_consensus.cc:2454] T d9eb216c97d441278a7446d729ed378b P 6a49c31e22d740fab75a5123639a3502 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate adadc55f535f4b0dbe3d04e41251d456 in term 0.
I20260812 06:55:00.874997 21816 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d9eb216c97d441278a7446d729ed378b" candidate_uuid: "adadc55f535f4b0dbe3d04e41251d456" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cf0e18c976a24d659861decac8e6b157" is_pre_election: true
I20260812 06:55:00.875245 21816 raft_consensus.cc:2454] T d9eb216c97d441278a7446d729ed378b P cf0e18c976a24d659861decac8e6b157 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate adadc55f535f4b0dbe3d04e41251d456 in term 0.
I20260812 06:55:00.875448 21901 leader_election.cc:302] T d9eb216c97d441278a7446d729ed378b P adadc55f535f4b0dbe3d04e41251d456 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a49c31e22d740fab75a5123639a3502, adadc55f535f4b0dbe3d04e41251d456; no voters:
I20260812 06:55:00.875747 22325 raft_consensus.cc:2781] T d9eb216c97d441278a7446d729ed378b P adadc55f535f4b0dbe3d04e41251d456 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 06:55:00.875828 22325 raft_consensus.cc:482] T d9eb216c97d441278a7446d729ed378b P adadc55f535f4b0dbe3d04e41251d456 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 06:55:00.875885 22325 raft_consensus.cc:3037] T d9eb216c97d441278a7446d729ed378b P adadc55f535f4b0dbe3d04e41251d456 [term 0 FOLLOWER]: Advancing to term 1
I20260812 06:55:00.877184 22325 raft_consensus.cc:504] T d9eb216c97d441278a7446d729ed378b P adadc55f535f4b0dbe3d04e41251d456 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf0e18c976a24d659861decac8e6b157" member_type: VOTER last_known_addr { host: "127.19.52.65" port: 45981 } } peers { permanent_uuid: "adadc55f535f4b0dbe3d04e41251d456" member_type: VOTER last_known_addr { host: "127.19.52.66" port: 36983 } } peers { permanent_uuid: "6a49c31e22d740fab75a5123639a3502" member_type: VOTER last_known_addr { host: "127.19.52.67" port: 44177 } }
I20260812 06:55:00.877581 22325 leader_election.cc:288] T d9eb216c97d441278a7446d729ed378b P adadc55f535f4b0dbe3d04e41251d456 [CANDIDATE]: Term 1 election: Requested vote from peers cf0e18c976a24d659861decac8e6b157 (127.19.52.65:45981), 6a49c31e22d740fab75a5123639a3502 (127.19.52.67:44177)
I20260812 06:55:00.878129 22140 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d9eb216c97d441278a7446d729ed378b" candidate_uuid: "adadc55f535f4b0dbe3d04e41251d456" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6a49c31e22d740fab75a5123639a3502"
I20260812 06:55:00.878121 21816 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d9eb216c97d441278a7446d729ed378b" candidate_uuid: "adadc55f535f4b0dbe3d04e41251d456" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cf0e18c976a24d659861decac8e6b157"
I20260812 06:55:00.878263 22140 raft_consensus.cc:3037] T d9eb216c97d441278a7446d729ed378b P 6a49c31e22d740fab75a5123639a3502 [term 0 FOLLOWER]: Advancing to term 1
I20260812 06:55:00.878263 21816 raft_consensus.cc:3037] T d9eb216c97d441278a7446d729ed378b P cf0e18c976a24d659861decac8e6b157 [term 0 FOLLOWER]: Advancing to term 1
I20260812 06:55:00.879483 21816 raft_consensus.cc:2454] T d9eb216c97d441278a7446d729ed378b P cf0e18c976a24d659861decac8e6b157 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate adadc55f535f4b0dbe3d04e41251d456 in term 1.
I20260812 06:55:00.879487 22140 raft_consensus.cc:2454] T d9eb216c97d441278a7446d729ed378b P 6a49c31e22d740fab75a5123639a3502 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate adadc55f535f4b0dbe3d04e41251d456 in term 1.
I20260812 06:55:00.879870 21898 leader_election.cc:302] T d9eb216c97d441278a7446d729ed378b P adadc55f535f4b0dbe3d04e41251d456 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 6a49c31e22d740fab75a5123639a3502, adadc55f535f4b0dbe3d04e41251d456, cf0e18c976a24d659861decac8e6b157; no voters:
I20260812 06:55:00.880139 22325 raft_consensus.cc:2781] T d9eb216c97d441278a7446d729ed378b P adadc55f535f4b0dbe3d04e41251d456 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 06:55:00.880417 22325 raft_consensus.cc:686] T d9eb216c97d441278a7446d729ed378b P adadc55f535f4b0dbe3d04e41251d456 [term 1 LEADER]: Becoming Leader. State: Replica: adadc55f535f4b0dbe3d04e41251d456, State: Running, Role: LEADER
I20260812 06:55:00.880741 22325 consensus_queue.cc:237] T d9eb216c97d441278a7446d729ed378b P adadc55f535f4b0dbe3d04e41251d456 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf0e18c976a24d659861decac8e6b157" member_type: VOTER last_known_addr { host: "127.19.52.65" port: 45981 } } peers { permanent_uuid: "adadc55f535f4b0dbe3d04e41251d456" member_type: VOTER last_known_addr { host: "127.19.52.66" port: 36983 } } peers { permanent_uuid: "6a49c31e22d740fab75a5123639a3502" member_type: VOTER last_known_addr { host: "127.19.52.67" port: 44177 } }
I20260812 06:55:00.883340 21633 catalog_manager.cc:5495] T d9eb216c97d441278a7446d729ed378b P adadc55f535f4b0dbe3d04e41251d456 reported cstate change: term changed from 0 to 1, leader changed from <none> to adadc55f535f4b0dbe3d04e41251d456 (127.19.52.66). New cstate: current_term: 1 leader_uuid: "adadc55f535f4b0dbe3d04e41251d456" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf0e18c976a24d659861decac8e6b157" member_type: VOTER last_known_addr { host: "127.19.52.65" port: 45981 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "adadc55f535f4b0dbe3d04e41251d456" member_type: VOTER last_known_addr { host: "127.19.52.66" port: 36983 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6a49c31e22d740fab75a5123639a3502" member_type: VOTER last_known_addr { host: "127.19.52.67" port: 44177 } health_report { overall_health: UNKNOWN } } }
I20260812 06:55:00.903137 22335 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:00.898874 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34672 (local address 127.19.52.66:36983)
0812 06:55:00.899232 (+ 358us) server_negotiation.cc:206] Beginning negotiation
0812 06:55:00.899240 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 06:55:00.899252 (+ 12us) server_negotiation.cc:407] Connection header received
0812 06:55:00.899333 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:55:00.899336 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:55:00.899441 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:55:00.899560 (+ 119us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 06:55:00.899859 (+ 299us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:00.900404 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:00.901380 (+ 976us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:00.901665 (+ 285us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:00.901807 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:55:00.902163 (+ 356us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 06:55:00.902698 (+ 535us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 06:55:00.902826 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0812 06:55:00.902898 (+ 72us) server_negotiation.cc:299] Negotiation successful
0812 06:55:00.902955 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":288,"thread_start_us":209,"threads_started":1}
I20260812 06:55:00.914556 22338 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:00.910790 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.65:45981 (local address 127.19.52.66:40591)
0812 06:55:00.911270 (+ 480us) negotiation.cc:107] Waiting for socket to connect
0812 06:55:00.911282 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 06:55:00.911426 (+ 144us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:55:00.911725 (+ 299us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:55:00.911729 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:55:00.911737 (+ 8us) client_negotiation.cc:189] Negotiated authn=TOKEN
0812 06:55:00.911876 (+ 139us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:55:00.911881 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:55:00.912656 (+ 775us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:55:00.912661 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:55:00.913282 (+ 621us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:55:00.913290 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:55:00.913766 (+ 476us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:55:00.913769 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:55:00.913871 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:55:00.913916 (+ 45us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0812 06:55:00.914241 (+ 325us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0812 06:55:00.914251 (+ 10us) client_negotiation.cc:769] Sending connection context
0812 06:55:00.914316 (+ 65us) client_negotiation.cc:240] Negotiation successful
0812 06:55:00.914399 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":362,"thread_start_us":154,"threads_started":1}
I20260812 06:55:00.914535 22329 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:00.910940 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.52.66:40591 (local address 127.19.52.65:45981)
0812 06:55:00.911181 (+ 241us) server_negotiation.cc:206] Beginning negotiation
0812 06:55:00.911189 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 06:55:00.911365 (+ 176us) server_negotiation.cc:407] Connection header received
0812 06:55:00.911563 (+ 198us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:55:00.911568 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:55:00.911622 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:55:00.911710 (+ 88us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 06:55:00.911989 (+ 279us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:00.912510 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:00.913410 (+ 900us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:00.913643 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:00.913757 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:55:00.914069 (+ 312us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 06:55:00.914151 (+ 82us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 06:55:00.914233 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0812 06:55:00.914325 (+ 92us) server_negotiation.cc:299] Negotiation successful
0812 06:55:00.914378 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":124}
I20260812 06:55:00.921135 22140 raft_consensus.cc:1261] T d9eb216c97d441278a7446d729ed378b P 6a49c31e22d740fab75a5123639a3502 [term 1 FOLLOWER]: Refusing update from remote peer adadc55f535f4b0dbe3d04e41251d456: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 06:55:00.921252 21816 raft_consensus.cc:1261] T d9eb216c97d441278a7446d729ed378b P cf0e18c976a24d659861decac8e6b157 [term 1 FOLLOWER]: Refusing update from remote peer adadc55f535f4b0dbe3d04e41251d456: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 06:55:00.921711 22332 consensus_queue.cc:1035] T d9eb216c97d441278a7446d729ed378b P adadc55f535f4b0dbe3d04e41251d456 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a49c31e22d740fab75a5123639a3502" member_type: VOTER last_known_addr { host: "127.19.52.67" port: 44177 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 06:55:00.921931 22325 consensus_queue.cc:1035] T d9eb216c97d441278a7446d729ed378b P adadc55f535f4b0dbe3d04e41251d456 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cf0e18c976a24d659861decac8e6b157" member_type: VOTER last_known_addr { host: "127.19.52.65" port: 45981 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 06:55:00.941311 22335 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:00.937792 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.52.65:44817 (local address 127.19.52.66:36983)
0812 06:55:00.937930 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0812 06:55:00.937936 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 06:55:00.938177 (+ 241us) server_negotiation.cc:407] Connection header received
0812 06:55:00.938299 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:55:00.938304 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:55:00.938369 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:55:00.938444 (+ 75us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 06:55:00.938715 (+ 271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:00.939224 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:00.940160 (+ 936us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:00.940412 (+ 252us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:00.940492 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:55:00.940822 (+ 330us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 06:55:00.940906 (+ 84us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 06:55:00.940988 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0812 06:55:00.941108 (+ 120us) server_negotiation.cc:299] Negotiation successful
0812 06:55:00.941165 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260812 06:55:00.941324 22343 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:00.937744 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.66:36983 (local address 127.19.52.65:44817)
0812 06:55:00.938053 (+ 309us) negotiation.cc:107] Waiting for socket to connect
0812 06:55:00.938068 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 06:55:00.938196 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:55:00.938481 (+ 285us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:55:00.938484 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:55:00.938492 (+ 8us) client_negotiation.cc:189] Negotiated authn=TOKEN
0812 06:55:00.938612 (+ 120us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:55:00.938618 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:55:00.939327 (+ 709us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:55:00.939330 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:55:00.940044 (+ 714us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:55:00.940053 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:55:00.940524 (+ 471us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:55:00.940528 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:55:00.940662 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:55:00.940693 (+ 31us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0812 06:55:00.941002 (+ 309us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0812 06:55:00.941010 (+ 8us) client_negotiation.cc:769] Sending connection context
0812 06:55:00.941094 (+ 84us) client_negotiation.cc:240] Negotiation successful
0812 06:55:00.941193 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":217,"thread_start_us":121,"threads_started":1}
I20260812 06:55:00.966478 21633 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:46644:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 06:55:00.966662 21633 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:46644:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 06:55:00.969116 21633 catalog_manager.cc:5782] T 00000000000000000000000000000000 P edbfe34f29a1474d96de7496f4a45b6f: Sending DeleteTablet for 3 replicas of tablet d9eb216c97d441278a7446d729ed378b
I20260812 06:55:00.969986 22117 tablet_service.cc:1514] Processing DeleteTablet for tablet d9eb216c97d441278a7446d729ed378b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 06:55:00 UTC) from {username='kudu', principal='kudu/127.19.52.126@KRBTEST.COM'} at 127.0.0.1:43562
I20260812 06:55:00.969995 21953 tablet_service.cc:1514] Processing DeleteTablet for tablet d9eb216c97d441278a7446d729ed378b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 06:55:00 UTC) from {username='kudu', principal='kudu/127.19.52.126@KRBTEST.COM'} at 127.0.0.1:34660
I20260812 06:55:00.970196 22306 tablet_replica.cc:330] T d9eb216c97d441278a7446d729ed378b P adadc55f535f4b0dbe3d04e41251d456: stopping tablet replica
I20260812 06:55:00.970197 22307 tablet_replica.cc:330] T d9eb216c97d441278a7446d729ed378b P 6a49c31e22d740fab75a5123639a3502: stopping tablet replica
I20260812 06:55:00.970324 22307 raft_consensus.cc:2229] T d9eb216c97d441278a7446d729ed378b P 6a49c31e22d740fab75a5123639a3502 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 06:55:00.970315 22306 raft_consensus.cc:2229] T d9eb216c97d441278a7446d729ed378b P adadc55f535f4b0dbe3d04e41251d456 [term 1 LEADER]: Raft consensus shutting down.
I20260812 06:55:00.970422 22307 raft_consensus.cc:2258] T d9eb216c97d441278a7446d729ed378b P 6a49c31e22d740fab75a5123639a3502 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 06:55:00.970638 22306 raft_consensus.cc:2258] T d9eb216c97d441278a7446d729ed378b P adadc55f535f4b0dbe3d04e41251d456 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 06:55:00.971091 22307 ts_tablet_manager.cc:1905] T d9eb216c97d441278a7446d729ed378b P 6a49c31e22d740fab75a5123639a3502: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 06:55:00.971931 22306 ts_tablet_manager.cc:1905] T d9eb216c97d441278a7446d729ed378b P adadc55f535f4b0dbe3d04e41251d456: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 06:55:00.973682 22307 ts_tablet_manager.cc:1918] T d9eb216c97d441278a7446d729ed378b P 6a49c31e22d740fab75a5123639a3502: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260812 06:55:00.973762 22307 log.cc:1196] T d9eb216c97d441278a7446d729ed378b P 6a49c31e22d740fab75a5123639a3502: Deleting WAL directory at /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786517687384214-19665-0/minicluster-data/ts-2/wal/wals/d9eb216c97d441278a7446d729ed378b
I20260812 06:55:00.974009 22307 ts_tablet_manager.cc:1939] T d9eb216c97d441278a7446d729ed378b P 6a49c31e22d740fab75a5123639a3502: Deleting consensus metadata
I20260812 06:55:00.974546 22306 ts_tablet_manager.cc:1918] T d9eb216c97d441278a7446d729ed378b P adadc55f535f4b0dbe3d04e41251d456: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260812 06:55:00.974877 21620 catalog_manager.cc:4841] TS 6a49c31e22d740fab75a5123639a3502 (127.19.52.67:44177): tablet d9eb216c97d441278a7446d729ed378b (table test-table [id=65a35ef676064a158d48153bfbfa55fb]) successfully deleted
I20260812 06:55:00.975361 21792 tablet_service.cc:1514] Processing DeleteTablet for tablet d9eb216c97d441278a7446d729ed378b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 06:55:00 UTC) from {username='kudu', principal='kudu/127.19.52.126@KRBTEST.COM'} at 127.0.0.1:47220
I20260812 06:55:00.975574 22305 tablet_replica.cc:330] T d9eb216c97d441278a7446d729ed378b P cf0e18c976a24d659861decac8e6b157: stopping tablet replica
I20260812 06:55:00.975708 22305 raft_consensus.cc:2229] T d9eb216c97d441278a7446d729ed378b P cf0e18c976a24d659861decac8e6b157 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 06:55:00.975834 22305 raft_consensus.cc:2258] T d9eb216c97d441278a7446d729ed378b P cf0e18c976a24d659861decac8e6b157 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 06:55:00.976179 21633 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:46644:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260812 06:55:00.974632 22306 log.cc:1196] T d9eb216c97d441278a7446d729ed378b P adadc55f535f4b0dbe3d04e41251d456: Deleting WAL directory at /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786517687384214-19665-0/minicluster-data/ts-1/wal/wals/d9eb216c97d441278a7446d729ed378b
W20260812 06:55:00.976691 21633 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 06:55:00.976941 22306 ts_tablet_manager.cc:1939] T d9eb216c97d441278a7446d729ed378b P adadc55f535f4b0dbe3d04e41251d456: Deleting consensus metadata
I20260812 06:55:00.977659 21620 catalog_manager.cc:4841] TS adadc55f535f4b0dbe3d04e41251d456 (127.19.52.66:36983): tablet d9eb216c97d441278a7446d729ed378b (table test-table [id=65a35ef676064a158d48153bfbfa55fb]) successfully deleted
I20260812 06:55:00.978229 22305 ts_tablet_manager.cc:1905] T d9eb216c97d441278a7446d729ed378b P cf0e18c976a24d659861decac8e6b157: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 06:55:00.981179 22305 ts_tablet_manager.cc:1918] T d9eb216c97d441278a7446d729ed378b P cf0e18c976a24d659861decac8e6b157: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260812 06:55:00.981276 22305 log.cc:1196] T d9eb216c97d441278a7446d729ed378b P cf0e18c976a24d659861decac8e6b157: Deleting WAL directory at /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786517687384214-19665-0/minicluster-data/ts-0/wal/wals/d9eb216c97d441278a7446d729ed378b
I20260812 06:55:00.981557 22305 ts_tablet_manager.cc:1939] T d9eb216c97d441278a7446d729ed378b P cf0e18c976a24d659861decac8e6b157: Deleting consensus metadata
I20260812 06:55:00.982529 21953 tablet_service.cc:1467] Processing CreateTablet for tablet e4cb8e6226f84b77916be7973d429e60 (DEFAULT_TABLE table=test-table [id=a2b6dff502714c3c8f4ffabd0d62041a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 06:55:00.982611 22117 tablet_service.cc:1467] Processing CreateTablet for tablet e4cb8e6226f84b77916be7973d429e60 (DEFAULT_TABLE table=test-table [id=a2b6dff502714c3c8f4ffabd0d62041a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 06:55:00.982748 21617 catalog_manager.cc:4841] TS cf0e18c976a24d659861decac8e6b157 (127.19.52.65:45981): tablet d9eb216c97d441278a7446d729ed378b (table test-table [id=65a35ef676064a158d48153bfbfa55fb]) successfully deleted
I20260812 06:55:00.982821 21953 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e4cb8e6226f84b77916be7973d429e60. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 06:55:00.982928 22117 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e4cb8e6226f84b77916be7973d429e60. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 06:55:00.985111 22317 tablet_bootstrap.cc:492] T e4cb8e6226f84b77916be7973d429e60 P 6a49c31e22d740fab75a5123639a3502: Bootstrap starting.
I20260812 06:55:00.985679 21792 tablet_service.cc:1467] Processing CreateTablet for tablet e4cb8e6226f84b77916be7973d429e60 (DEFAULT_TABLE table=test-table [id=a2b6dff502714c3c8f4ffabd0d62041a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 06:55:00.985953 21792 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e4cb8e6226f84b77916be7973d429e60. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 06:55:00.986097 22317 tablet_bootstrap.cc:654] T e4cb8e6226f84b77916be7973d429e60 P 6a49c31e22d740fab75a5123639a3502: Neither blocks nor log segments found. Creating new log.
I20260812 06:55:00.987473 22317 tablet_bootstrap.cc:492] T e4cb8e6226f84b77916be7973d429e60 P 6a49c31e22d740fab75a5123639a3502: No bootstrap required, opened a new log
I20260812 06:55:00.987594 22317 ts_tablet_manager.cc:1397] T e4cb8e6226f84b77916be7973d429e60 P 6a49c31e22d740fab75a5123639a3502: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260812 06:55:00.988013 22317 raft_consensus.cc:348] T e4cb8e6226f84b77916be7973d429e60 P 6a49c31e22d740fab75a5123639a3502 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf0e18c976a24d659861decac8e6b157" member_type: VOTER last_known_addr { host: "127.19.52.65" port: 45981 } } peers { permanent_uuid: "adadc55f535f4b0dbe3d04e41251d456" member_type: VOTER last_known_addr { host: "127.19.52.66" port: 36983 } } peers { permanent_uuid: "6a49c31e22d740fab75a5123639a3502" member_type: VOTER last_known_addr { host: "127.19.52.67" port: 44177 } }
I20260812 06:55:00.988144 22317 raft_consensus.cc:374] T e4cb8e6226f84b77916be7973d429e60 P 6a49c31e22d740fab75a5123639a3502 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 06:55:00.988188 22317 raft_consensus.cc:729] T e4cb8e6226f84b77916be7973d429e60 P 6a49c31e22d740fab75a5123639a3502 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6a49c31e22d740fab75a5123639a3502, State: Initialized, Role: FOLLOWER
I20260812 06:55:00.988339 22321 tablet_bootstrap.cc:492] T e4cb8e6226f84b77916be7973d429e60 P adadc55f535f4b0dbe3d04e41251d456: Bootstrap starting.
I20260812 06:55:00.988353 22317 consensus_queue.cc:260] T e4cb8e6226f84b77916be7973d429e60 P 6a49c31e22d740fab75a5123639a3502 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf0e18c976a24d659861decac8e6b157" member_type: VOTER last_known_addr { host: "127.19.52.65" port: 45981 } } peers { permanent_uuid: "adadc55f535f4b0dbe3d04e41251d456" member_type: VOTER last_known_addr { host: "127.19.52.66" port: 36983 } } peers { permanent_uuid: "6a49c31e22d740fab75a5123639a3502" member_type: VOTER last_known_addr { host: "127.19.52.67" port: 44177 } }
I20260812 06:55:00.989171 22317 ts_tablet_manager.cc:1428] T e4cb8e6226f84b77916be7973d429e60 P 6a49c31e22d740fab75a5123639a3502: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260812 06:55:00.989404 22321 tablet_bootstrap.cc:654] T e4cb8e6226f84b77916be7973d429e60 P adadc55f535f4b0dbe3d04e41251d456: Neither blocks nor log segments found. Creating new log.
I20260812 06:55:00.990674 22321 tablet_bootstrap.cc:492] T e4cb8e6226f84b77916be7973d429e60 P adadc55f535f4b0dbe3d04e41251d456: No bootstrap required, opened a new log
I20260812 06:55:00.990796 22321 ts_tablet_manager.cc:1397] T e4cb8e6226f84b77916be7973d429e60 P adadc55f535f4b0dbe3d04e41251d456: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260812 06:55:00.991326 22321 raft_consensus.cc:348] T e4cb8e6226f84b77916be7973d429e60 P adadc55f535f4b0dbe3d04e41251d456 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf0e18c976a24d659861decac8e6b157" member_type: VOTER last_known_addr { host: "127.19.52.65" port: 45981 } } peers { permanent_uuid: "adadc55f535f4b0dbe3d04e41251d456" member_type: VOTER last_known_addr { host: "127.19.52.66" port: 36983 } } peers { permanent_uuid: "6a49c31e22d740fab75a5123639a3502" member_type: VOTER last_known_addr { host: "127.19.52.67" port: 44177 } }
I20260812 06:55:00.991459 22321 raft_consensus.cc:374] T e4cb8e6226f84b77916be7973d429e60 P adadc55f535f4b0dbe3d04e41251d456 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 06:55:00.991503 22321 raft_consensus.cc:729] T e4cb8e6226f84b77916be7973d429e60 P adadc55f535f4b0dbe3d04e41251d456 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: adadc55f535f4b0dbe3d04e41251d456, State: Initialized, Role: FOLLOWER
I20260812 06:55:00.991649 22321 consensus_queue.cc:260] T e4cb8e6226f84b77916be7973d429e60 P adadc55f535f4b0dbe3d04e41251d456 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf0e18c976a24d659861decac8e6b157" member_type: VOTER last_known_addr { host: "127.19.52.65" port: 45981 } } peers { permanent_uuid: "adadc55f535f4b0dbe3d04e41251d456" member_type: VOTER last_known_addr { host: "127.19.52.66" port: 36983 } } peers { permanent_uuid: "6a49c31e22d740fab75a5123639a3502" member_type: VOTER last_known_addr { host: "127.19.52.67" port: 44177 } }
I20260812 06:55:00.991945 22321 ts_tablet_manager.cc:1428] T e4cb8e6226f84b77916be7973d429e60 P adadc55f535f4b0dbe3d04e41251d456: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20260812 06:55:00.992312 22318 tablet_bootstrap.cc:492] T e4cb8e6226f84b77916be7973d429e60 P cf0e18c976a24d659861decac8e6b157: Bootstrap starting.
I20260812 06:55:00.993273 22318 tablet_bootstrap.cc:654] T e4cb8e6226f84b77916be7973d429e60 P cf0e18c976a24d659861decac8e6b157: Neither blocks nor log segments found. Creating new log.
I20260812 06:55:00.994946 22318 tablet_bootstrap.cc:492] T e4cb8e6226f84b77916be7973d429e60 P cf0e18c976a24d659861decac8e6b157: No bootstrap required, opened a new log
I20260812 06:55:00.995041 22318 ts_tablet_manager.cc:1397] T e4cb8e6226f84b77916be7973d429e60 P cf0e18c976a24d659861decac8e6b157: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.001s
I20260812 06:55:00.995437 22318 raft_consensus.cc:348] T e4cb8e6226f84b77916be7973d429e60 P cf0e18c976a24d659861decac8e6b157 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf0e18c976a24d659861decac8e6b157" member_type: VOTER last_known_addr { host: "127.19.52.65" port: 45981 } } peers { permanent_uuid: "adadc55f535f4b0dbe3d04e41251d456" member_type: VOTER last_known_addr { host: "127.19.52.66" port: 36983 } } peers { permanent_uuid: "6a49c31e22d740fab75a5123639a3502" member_type: VOTER last_known_addr { host: "127.19.52.67" port: 44177 } }
I20260812 06:55:00.995561 22318 raft_consensus.cc:374] T e4cb8e6226f84b77916be7973d429e60 P cf0e18c976a24d659861decac8e6b157 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 06:55:00.995594 22318 raft_consensus.cc:729] T e4cb8e6226f84b77916be7973d429e60 P cf0e18c976a24d659861decac8e6b157 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cf0e18c976a24d659861decac8e6b157, State: Initialized, Role: FOLLOWER
I20260812 06:55:00.995719 22318 consensus_queue.cc:260] T e4cb8e6226f84b77916be7973d429e60 P cf0e18c976a24d659861decac8e6b157 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf0e18c976a24d659861decac8e6b157" member_type: VOTER last_known_addr { host: "127.19.52.65" port: 45981 } } peers { permanent_uuid: "adadc55f535f4b0dbe3d04e41251d456" member_type: VOTER last_known_addr { host: "127.19.52.66" port: 36983 } } peers { permanent_uuid: "6a49c31e22d740fab75a5123639a3502" member_type: VOTER last_known_addr { host: "127.19.52.67" port: 44177 } }
I20260812 06:55:00.995981 22318 ts_tablet_manager.cc:1428] T e4cb8e6226f84b77916be7973d429e60 P cf0e18c976a24d659861decac8e6b157: Time spent starting tablet: real 0.001s user 0.001s sys 0.001s
I20260812 06:55:01.112108 22323 raft_consensus.cc:482] T e4cb8e6226f84b77916be7973d429e60 P 6a49c31e22d740fab75a5123639a3502 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 06:55:01.112282 22323 raft_consensus.cc:504] T e4cb8e6226f84b77916be7973d429e60 P 6a49c31e22d740fab75a5123639a3502 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf0e18c976a24d659861decac8e6b157" member_type: VOTER last_known_addr { host: "127.19.52.65" port: 45981 } } peers { permanent_uuid: "adadc55f535f4b0dbe3d04e41251d456" member_type: VOTER last_known_addr { host: "127.19.52.66" port: 36983 } } peers { permanent_uuid: "6a49c31e22d740fab75a5123639a3502" member_type: VOTER last_known_addr { host: "127.19.52.67" port: 44177 } }
I20260812 06:55:01.113564 22323 leader_election.cc:288] T e4cb8e6226f84b77916be7973d429e60 P 6a49c31e22d740fab75a5123639a3502 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cf0e18c976a24d659861decac8e6b157 (127.19.52.65:45981), adadc55f535f4b0dbe3d04e41251d456 (127.19.52.66:36983)
I20260812 06:55:01.118521 22329 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:01.113986 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.52.67:53567 (local address 127.19.52.65:45981)
0812 06:55:01.114120 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0812 06:55:01.114126 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 06:55:01.114388 (+ 262us) server_negotiation.cc:407] Connection header received
0812 06:55:01.114574 (+ 186us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:55:01.114577 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:55:01.114649 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:55:01.114738 (+ 89us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 06:55:01.115054 (+ 316us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:01.115607 (+ 553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:01.117147 (+ 1540us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:01.117783 (+ 636us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:01.117926 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:55:01.117981 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0812 06:55:01.118275 (+ 294us) server_negotiation.cc:299] Negotiation successful
0812 06:55:01.118375 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260812 06:55:01.118580 22335 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:01.114080 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.52.67:53899 (local address 127.19.52.66:36983)
0812 06:55:01.114218 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0812 06:55:01.114222 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 06:55:01.114414 (+ 192us) server_negotiation.cc:407] Connection header received
0812 06:55:01.114665 (+ 251us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:55:01.114668 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:55:01.114725 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:55:01.114795 (+ 70us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 06:55:01.115170 (+ 375us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:01.116064 (+ 894us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:01.117417 (+ 1353us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:01.117865 (+ 448us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:01.117994 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:55:01.118050 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0812 06:55:01.118358 (+ 308us) server_negotiation.cc:299] Negotiation successful
0812 06:55:01.118445 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260812 06:55:01.119056 22350 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:01.113773 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.65:45981 (local address 127.19.52.67:53567)
0812 06:55:01.114258 (+ 485us) negotiation.cc:107] Waiting for socket to connect
0812 06:55:01.114275 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0812 06:55:01.114443 (+ 168us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:55:01.114771 (+ 328us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:55:01.114774 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:55:01.114791 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 06:55:01.114921 (+ 130us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:55:01.114927 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:55:01.115754 (+ 827us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:55:01.115758 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:55:01.116998 (+ 1240us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:55:01.117009 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:55:01.117927 (+ 918us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:55:01.117933 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:55:01.118058 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:55:01.118091 (+ 33us) client_negotiation.cc:769] Sending connection context
0812 06:55:01.118780 (+ 689us) client_negotiation.cc:240] Negotiation successful
0812 06:55:01.118877 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":410,"mutex_wait_us":53,"thread_start_us":95,"threads_started":1}
I20260812 06:55:01.119633 21816 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e4cb8e6226f84b77916be7973d429e60" candidate_uuid: "6a49c31e22d740fab75a5123639a3502" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cf0e18c976a24d659861decac8e6b157" is_pre_election: true
I20260812 06:55:01.119719 22351 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:01.113873 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.66:36983 (local address 127.19.52.67:53899)
0812 06:55:01.114321 (+ 448us) negotiation.cc:107] Waiting for socket to connect
0812 06:55:01.114336 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 06:55:01.114503 (+ 167us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:55:01.114881 (+ 378us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:55:01.114884 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:55:01.114902 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 06:55:01.115044 (+ 142us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:55:01.115051 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:55:01.116219 (+ 1168us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:55:01.116223 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:55:01.117298 (+ 1075us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:55:01.117306 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:55:01.118087 (+ 781us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:55:01.118091 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:55:01.118209 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:55:01.118232 (+ 23us) client_negotiation.cc:769] Sending connection context
0812 06:55:01.119094 (+ 862us) client_negotiation.cc:240] Negotiation successful
0812 06:55:01.119532 (+ 438us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":379,"mutex_wait_us":1,"thread_start_us":116,"threads_started":1}
I20260812 06:55:01.119827 21816 raft_consensus.cc:2454] T e4cb8e6226f84b77916be7973d429e60 P cf0e18c976a24d659861decac8e6b157 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a49c31e22d740fab75a5123639a3502 in term 0.
I20260812 06:55:01.120196 21980 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e4cb8e6226f84b77916be7973d429e60" candidate_uuid: "6a49c31e22d740fab75a5123639a3502" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "adadc55f535f4b0dbe3d04e41251d456" is_pre_election: true
I20260812 06:55:01.120239 22063 leader_election.cc:302] T e4cb8e6226f84b77916be7973d429e60 P 6a49c31e22d740fab75a5123639a3502 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a49c31e22d740fab75a5123639a3502, cf0e18c976a24d659861decac8e6b157; no voters:
I20260812 06:55:01.120344 21980 raft_consensus.cc:2454] T e4cb8e6226f84b77916be7973d429e60 P adadc55f535f4b0dbe3d04e41251d456 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a49c31e22d740fab75a5123639a3502 in term 0.
I20260812 06:55:01.120669 22323 raft_consensus.cc:2781] T e4cb8e6226f84b77916be7973d429e60 P 6a49c31e22d740fab75a5123639a3502 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 06:55:01.120757 22323 raft_consensus.cc:482] T e4cb8e6226f84b77916be7973d429e60 P 6a49c31e22d740fab75a5123639a3502 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 06:55:01.120810 22323 raft_consensus.cc:3037] T e4cb8e6226f84b77916be7973d429e60 P 6a49c31e22d740fab75a5123639a3502 [term 0 FOLLOWER]: Advancing to term 1
I20260812 06:55:01.121889 22323 raft_consensus.cc:504] T e4cb8e6226f84b77916be7973d429e60 P 6a49c31e22d740fab75a5123639a3502 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf0e18c976a24d659861decac8e6b157" member_type: VOTER last_known_addr { host: "127.19.52.65" port: 45981 } } peers { permanent_uuid: "adadc55f535f4b0dbe3d04e41251d456" member_type: VOTER last_known_addr { host: "127.19.52.66" port: 36983 } } peers { permanent_uuid: "6a49c31e22d740fab75a5123639a3502" member_type: VOTER last_known_addr { host: "127.19.52.67" port: 44177 } }
I20260812 06:55:01.122298 22323 leader_election.cc:288] T e4cb8e6226f84b77916be7973d429e60 P 6a49c31e22d740fab75a5123639a3502 [CANDIDATE]: Term 1 election: Requested vote from peers cf0e18c976a24d659861decac8e6b157 (127.19.52.65:45981), adadc55f535f4b0dbe3d04e41251d456 (127.19.52.66:36983)
I20260812 06:55:01.122668 21980 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e4cb8e6226f84b77916be7973d429e60" candidate_uuid: "6a49c31e22d740fab75a5123639a3502" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "adadc55f535f4b0dbe3d04e41251d456"
I20260812 06:55:01.122634 21816 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e4cb8e6226f84b77916be7973d429e60" candidate_uuid: "6a49c31e22d740fab75a5123639a3502" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cf0e18c976a24d659861decac8e6b157"
I20260812 06:55:01.122781 21816 raft_consensus.cc:3037] T e4cb8e6226f84b77916be7973d429e60 P cf0e18c976a24d659861decac8e6b157 [term 0 FOLLOWER]: Advancing to term 1
I20260812 06:55:01.122805 21980 raft_consensus.cc:3037] T e4cb8e6226f84b77916be7973d429e60 P adadc55f535f4b0dbe3d04e41251d456 [term 0 FOLLOWER]: Advancing to term 1
I20260812 06:55:01.123769 21980 raft_consensus.cc:2454] T e4cb8e6226f84b77916be7973d429e60 P adadc55f535f4b0dbe3d04e41251d456 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a49c31e22d740fab75a5123639a3502 in term 1.
I20260812 06:55:01.123996 21816 raft_consensus.cc:2454] T e4cb8e6226f84b77916be7973d429e60 P cf0e18c976a24d659861decac8e6b157 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a49c31e22d740fab75a5123639a3502 in term 1.
I20260812 06:55:01.124836 22066 leader_election.cc:302] T e4cb8e6226f84b77916be7973d429e60 P 6a49c31e22d740fab75a5123639a3502 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a49c31e22d740fab75a5123639a3502, adadc55f535f4b0dbe3d04e41251d456; no voters:
I20260812 06:55:01.125057 22323 raft_consensus.cc:2781] T e4cb8e6226f84b77916be7973d429e60 P 6a49c31e22d740fab75a5123639a3502 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 06:55:01.125295 22323 raft_consensus.cc:686] T e4cb8e6226f84b77916be7973d429e60 P 6a49c31e22d740fab75a5123639a3502 [term 1 LEADER]: Becoming Leader. State: Replica: 6a49c31e22d740fab75a5123639a3502, State: Running, Role: LEADER
I20260812 06:55:01.125583 22323 consensus_queue.cc:237] T e4cb8e6226f84b77916be7973d429e60 P 6a49c31e22d740fab75a5123639a3502 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf0e18c976a24d659861decac8e6b157" member_type: VOTER last_known_addr { host: "127.19.52.65" port: 45981 } } peers { permanent_uuid: "adadc55f535f4b0dbe3d04e41251d456" member_type: VOTER last_known_addr { host: "127.19.52.66" port: 36983 } } peers { permanent_uuid: "6a49c31e22d740fab75a5123639a3502" member_type: VOTER last_known_addr { host: "127.19.52.67" port: 44177 } }
I20260812 06:55:01.128369 21633 catalog_manager.cc:5495] T e4cb8e6226f84b77916be7973d429e60 P 6a49c31e22d740fab75a5123639a3502 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6a49c31e22d740fab75a5123639a3502 (127.19.52.67). New cstate: current_term: 1 leader_uuid: "6a49c31e22d740fab75a5123639a3502" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf0e18c976a24d659861decac8e6b157" member_type: VOTER last_known_addr { host: "127.19.52.65" port: 45981 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "adadc55f535f4b0dbe3d04e41251d456" member_type: VOTER last_known_addr { host: "127.19.52.66" port: 36983 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6a49c31e22d740fab75a5123639a3502" member_type: VOTER last_known_addr { host: "127.19.52.67" port: 44177 } health_report { overall_health: HEALTHY } } }
I20260812 06:55:01.163456 22328 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:01.159384 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43570 (local address 127.19.52.67:44177)
0812 06:55:01.159531 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0812 06:55:01.159537 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 06:55:01.159570 (+ 33us) server_negotiation.cc:407] Connection header received
0812 06:55:01.159777 (+ 207us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:55:01.159780 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:55:01.159846 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:55:01.159963 (+ 117us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 06:55:01.160239 (+ 276us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:01.160780 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:01.161691 (+ 911us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:01.161923 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:01.162039 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:55:01.162372 (+ 333us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 06:55:01.162985 (+ 613us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 06:55:01.163086 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0812 06:55:01.163210 (+ 124us) server_negotiation.cc:299] Negotiation successful
0812 06:55:01.163262 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260812 06:55:01.174372 22329 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:01.171295 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.52.67:48029 (local address 127.19.52.65:45981)
0812 06:55:01.171473 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0812 06:55:01.171477 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 06:55:01.171492 (+ 15us) server_negotiation.cc:407] Connection header received
0812 06:55:01.171595 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:55:01.171597 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:55:01.171640 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:55:01.171704 (+ 64us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 06:55:01.171960 (+ 256us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:01.172469 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:01.173399 (+ 930us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:01.173627 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:01.173708 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:55:01.173987 (+ 279us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 06:55:01.174062 (+ 75us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 06:55:01.174128 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0812 06:55:01.174211 (+ 83us) server_negotiation.cc:299] Negotiation successful
0812 06:55:01.174259 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260812 06:55:01.174407 22357 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:01.170987 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.65:45981 (local address 127.19.52.67:48029)
0812 06:55:01.171396 (+ 409us) negotiation.cc:107] Waiting for socket to connect
0812 06:55:01.171411 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 06:55:01.171505 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:55:01.171732 (+ 227us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:55:01.171735 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:55:01.171741 (+ 6us) client_negotiation.cc:189] Negotiated authn=TOKEN
0812 06:55:01.171840 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:55:01.171845 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:55:01.172592 (+ 747us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:55:01.172594 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:55:01.173290 (+ 696us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:55:01.173300 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:55:01.173728 (+ 428us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:55:01.173731 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:55:01.173797 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:55:01.173838 (+ 41us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0812 06:55:01.174148 (+ 310us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0812 06:55:01.174153 (+ 5us) client_negotiation.cc:769] Sending connection context
0812 06:55:01.174203 (+ 50us) client_negotiation.cc:240] Negotiation successful
0812 06:55:01.174268 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":294,"thread_start_us":126,"threads_started":1}
I20260812 06:55:01.180603 21980 raft_consensus.cc:1261] T e4cb8e6226f84b77916be7973d429e60 P adadc55f535f4b0dbe3d04e41251d456 [term 1 FOLLOWER]: Refusing update from remote peer 6a49c31e22d740fab75a5123639a3502: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 06:55:01.180758 21816 raft_consensus.cc:1261] T e4cb8e6226f84b77916be7973d429e60 P cf0e18c976a24d659861decac8e6b157 [term 1 FOLLOWER]: Refusing update from remote peer 6a49c31e22d740fab75a5123639a3502: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 06:55:01.181083 22353 consensus_queue.cc:1035] T e4cb8e6226f84b77916be7973d429e60 P 6a49c31e22d740fab75a5123639a3502 [LEADER]: Connected to new peer: Peer: permanent_uuid: "adadc55f535f4b0dbe3d04e41251d456" member_type: VOTER last_known_addr { host: "127.19.52.66" port: 36983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 06:55:01.181422 22353 consensus_queue.cc:1035] T e4cb8e6226f84b77916be7973d429e60 P 6a49c31e22d740fab75a5123639a3502 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cf0e18c976a24d659861decac8e6b157" member_type: VOTER last_known_addr { host: "127.19.52.65" port: 45981 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 06:55:01.201097 22328 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:01.196945 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.52.65:46821 (local address 127.19.52.67:44177)
0812 06:55:01.197379 (+ 434us) server_negotiation.cc:206] Beginning negotiation
0812 06:55:01.197383 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 06:55:01.197397 (+ 14us) server_negotiation.cc:407] Connection header received
0812 06:55:01.197445 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:55:01.197448 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:55:01.197502 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:55:01.197586 (+ 84us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 06:55:01.197831 (+ 245us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:01.198639 (+ 808us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:01.199767 (+ 1128us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:01.200149 (+ 382us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:01.200249 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:55:01.200653 (+ 404us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 06:55:01.200772 (+ 119us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 06:55:01.200848 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0812 06:55:01.200942 (+ 94us) server_negotiation.cc:299] Negotiation successful
0812 06:55:01.200997 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":345}
I20260812 06:55:01.201174 22343 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:01.196654 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.67:44177 (local address 127.19.52.65:46821)
0812 06:55:01.196991 (+ 337us) negotiation.cc:107] Waiting for socket to connect
0812 06:55:01.197009 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0812 06:55:01.197088 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:55:01.197621 (+ 533us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:55:01.197623 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:55:01.197630 (+ 7us) client_negotiation.cc:189] Negotiated authn=TOKEN
0812 06:55:01.197719 (+ 89us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:55:01.197724 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:55:01.198799 (+ 1075us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:55:01.198802 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:55:01.199647 (+ 845us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:55:01.199657 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:55:01.200333 (+ 676us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:55:01.200336 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:55:01.200426 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:55:01.200462 (+ 36us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0812 06:55:01.200854 (+ 392us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0812 06:55:01.200861 (+ 7us) client_negotiation.cc:769] Sending connection context
0812 06:55:01.200934 (+ 73us) client_negotiation.cc:240] Negotiation successful
0812 06:55:01.201031 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":227}
W20260812 06:55:01.201920 22287 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260812 06:55:02.773432 21632 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:46644:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 06:55:02.773603 21632 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:46644:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 06:55:02.776446 21632 catalog_manager.cc:5782] T 00000000000000000000000000000000 P edbfe34f29a1474d96de7496f4a45b6f: Sending DeleteTablet for 3 replicas of tablet e4cb8e6226f84b77916be7973d429e60
I20260812 06:55:02.777213 21792 tablet_service.cc:1514] Processing DeleteTablet for tablet e4cb8e6226f84b77916be7973d429e60 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 06:55:02 UTC) from {username='kudu', principal='kudu/127.19.52.126@KRBTEST.COM'} at 127.0.0.1:47220
I20260812 06:55:02.777213 21953 tablet_service.cc:1514] Processing DeleteTablet for tablet e4cb8e6226f84b77916be7973d429e60 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 06:55:02 UTC) from {username='kudu', principal='kudu/127.19.52.126@KRBTEST.COM'} at 127.0.0.1:34660
I20260812 06:55:02.777375 22117 tablet_service.cc:1514] Processing DeleteTablet for tablet e4cb8e6226f84b77916be7973d429e60 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 06:55:02 UTC) from {username='kudu', principal='kudu/127.19.52.126@KRBTEST.COM'} at 127.0.0.1:43562
I20260812 06:55:02.777609 22395 tablet_replica.cc:330] T e4cb8e6226f84b77916be7973d429e60 P cf0e18c976a24d659861decac8e6b157: stopping tablet replica
I20260812 06:55:02.777748 22395 raft_consensus.cc:2229] T e4cb8e6226f84b77916be7973d429e60 P cf0e18c976a24d659861decac8e6b157 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 06:55:02.777848 22397 tablet_replica.cc:330] T e4cb8e6226f84b77916be7973d429e60 P 6a49c31e22d740fab75a5123639a3502: stopping tablet replica
I20260812 06:55:02.777882 22395 raft_consensus.cc:2258] T e4cb8e6226f84b77916be7973d429e60 P cf0e18c976a24d659861decac8e6b157 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 06:55:02.777985 19665 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu with pid 21719
I20260812 06:55:02.778018 22397 raft_consensus.cc:2229] T e4cb8e6226f84b77916be7973d429e60 P 6a49c31e22d740fab75a5123639a3502 [term 1 LEADER]: Raft consensus shutting down.
I20260812 06:55:02.781577 22397 raft_consensus.cc:2258] T e4cb8e6226f84b77916be7973d429e60 P 6a49c31e22d740fab75a5123639a3502 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 06:55:02.782428 22397 ts_tablet_manager.cc:1905] T e4cb8e6226f84b77916be7973d429e60 P 6a49c31e22d740fab75a5123639a3502: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 06:55:02.784925 22397 ts_tablet_manager.cc:1918] T e4cb8e6226f84b77916be7973d429e60 P 6a49c31e22d740fab75a5123639a3502: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260812 06:55:02.785000 22397 log.cc:1196] T e4cb8e6226f84b77916be7973d429e60 P 6a49c31e22d740fab75a5123639a3502: Deleting WAL directory at /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786517687384214-19665-0/minicluster-data/ts-2/wal/wals/e4cb8e6226f84b77916be7973d429e60
I20260812 06:55:02.785319 22397 ts_tablet_manager.cc:1939] T e4cb8e6226f84b77916be7973d429e60 P 6a49c31e22d740fab75a5123639a3502: Deleting consensus metadata
I20260812 06:55:02.785343 22396 tablet_replica.cc:330] T e4cb8e6226f84b77916be7973d429e60 P adadc55f535f4b0dbe3d04e41251d456: stopping tablet replica
I20260812 06:55:02.785511 22396 raft_consensus.cc:2229] T e4cb8e6226f84b77916be7973d429e60 P adadc55f535f4b0dbe3d04e41251d456 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 06:55:02.785657 22396 raft_consensus.cc:2258] T e4cb8e6226f84b77916be7973d429e60 P adadc55f535f4b0dbe3d04e41251d456 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 06:55:02.786237 21620 catalog_manager.cc:4841] TS 6a49c31e22d740fab75a5123639a3502 (127.19.52.67:44177): tablet e4cb8e6226f84b77916be7973d429e60 (table test-table [id=a2b6dff502714c3c8f4ffabd0d62041a]) successfully deleted
I20260812 06:55:02.786415 22396 ts_tablet_manager.cc:1905] T e4cb8e6226f84b77916be7973d429e60 P adadc55f535f4b0dbe3d04e41251d456: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 06:55:02.788645 22396 ts_tablet_manager.cc:1918] T e4cb8e6226f84b77916be7973d429e60 P adadc55f535f4b0dbe3d04e41251d456: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260812 06:55:02.788741 22396 log.cc:1196] T e4cb8e6226f84b77916be7973d429e60 P adadc55f535f4b0dbe3d04e41251d456: Deleting WAL directory at /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786517687384214-19665-0/minicluster-data/ts-1/wal/wals/e4cb8e6226f84b77916be7973d429e60
I20260812 06:55:02.789002 22396 ts_tablet_manager.cc:1939] T e4cb8e6226f84b77916be7973d429e60 P adadc55f535f4b0dbe3d04e41251d456: Deleting consensus metadata
W20260812 06:55:02.789992 21617 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.19.52.65:45981 (error 108)
I20260812 06:55:02.790501 21620 catalog_manager.cc:4841] TS adadc55f535f4b0dbe3d04e41251d456 (127.19.52.66:36983): tablet e4cb8e6226f84b77916be7973d429e60 (table test-table [id=a2b6dff502714c3c8f4ffabd0d62041a]) successfully deleted
I20260812 06:55:02.790915 19665 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu with pid 21879
I20260812 06:55:02.791322 22400 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:02.790805 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.65:45981 (local address 127.0.0.1:47254)
0812 06:55:02.791107 (+ 302us) negotiation.cc:107] Waiting for socket to connect
0812 06:55:02.791198 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.52.65:45981: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":204,"thread_start_us":96,"threads_started":1}
W20260812 06:55:02.792214 21617 catalog_manager.cc:4584] TS cf0e18c976a24d659861decac8e6b157 (127.19.52.65:45981): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet e4cb8e6226f84b77916be7973d429e60: Network error: Client connection negotiation failed: client connection to 127.19.52.65:45981: connect: Connection refused (error 111)
I20260812 06:55:02.800103 19665 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu with pid 22044
I20260812 06:55:02.810750 19665 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu with pid 21600
2026-08-12T06:55:02Z chronyd exiting
[ OK ] SecurityITest.TxnSmokeWithDifferentUserTypes (6876 ms)
[ RUN ] SecurityITest.TestNoKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786517687384214-19665-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 06:55:02 dist-test-slave-65mx krb5kdc[22404](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 06:55:02 dist-test-slave-65mx krb5kdc[22404](info): set up 2 sockets
Aug 12 06:55:02 dist-test-slave-65mx krb5kdc[22404](info): commencing operation
krb5kdc: starting...
W20260812 06:55:03.992754 19665 mini_kdc.cc:121] Time spent starting KDC: real 1.154s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786517687384214-19665-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786517687384214-19665-0/krb5kdc/test-user.keytab.
Aug 12 06:55:04 dist-test-slave-65mx krb5kdc[22404](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517704, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T06:55:04Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T06:55:04Z Disabled control of system clock
WARNING: no policy specified for kudu/127.19.52.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.52.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.52.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786517687384214-19665-0/krb5kdc/kudu_127.19.52.126.keytab.
Entry for principal kudu/127.19.52.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786517687384214-19665-0/krb5kdc/kudu_127.19.52.126.keytab.
I20260812 06:55:04.138350 19665 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu
/tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786517687384214-19665-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786517687384214-19665-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786517687384214-19665-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786517687384214-19665-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.52.126:36197
--webserver_interface=127.19.52.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786517687384214-19665-0/krb5kdc/kudu_127.19.52.126.keytab
--principal=kudu/127.19.52.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.52.84:43891
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.19.52.126:36197
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786517687384214-19665-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786517687384214-19665-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786517687384214-19665-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 06:55:04.257412 22417 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:55:04.257759 22417 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:55:04.257831 22417 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 06:55:04.262035 22417 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 06:55:04.262131 22417 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 06:55:04.262181 22417 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 06:55:04.262236 22417 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 06:55:04.262276 22417 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 06:55:04.267772 22417 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.52.84:43891
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786517687384214-19665-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786517687384214-19665-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.19.52.126:36197
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786517687384214-19665-0/krb5kdc/kudu_127.19.52.126.keytab
--principal=kudu/127.19.52.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.52.126:36197
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786517687384214-19665-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.19.52.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22417
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786517687384214-19665-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 06:15:54 UTC on ccycloud.quasar-rnfzce.root.comops.site
build id 4581989
I20260812 06:55:04.269060 22417 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:55:04.270149 22417 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 06:55:04.279702 22428 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:55:04.279740 22425 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:55:04.279709 22426 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:55:04.279752 22417 server_base.cc:1034] running on GCE node
I20260812 06:55:04.280800 22417 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:55:04.282023 22417 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:55:04.283209 22417 hybrid_clock.cc:648] HybridClock initialized: now 1786517704283176 us; error 46 us; skew 500 ppm
Aug 12 06:55:04 dist-test-slave-65mx krb5kdc[22404](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517704, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.52.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 06:55:04.286295 22417 init.cc:377] Logged in from keytab as kudu/127.19.52.126@KRBTEST.COM (short username kudu)
I20260812 06:55:04.287323 22417 webserver.cc:460] Webserver started at http://127.19.52.126:40213/ using document root <none> and password file <none>
I20260812 06:55:04.287927 22417 fs_manager.cc:359] Metadata directory not provided
I20260812 06:55:04.288021 22417 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:55:04.288277 22417 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 06:55:04.290462 22417 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786517687384214-19665-0/minicluster-data/master-0/data/instance:
uuid: "7eb082d4a2dd42cda806c46bcf021401"
format_stamp: "Formatted at 2026-08-12 06:55:04 on dist-test-slave-65mx"
server_key: "561cc1cf26890800513fae9e042e5eac"
server_key_iv: "d75b295fc7a9347c5e37512216ca07b9"
server_key_version: "encryptionkey@0"
I20260812 06:55:04.291059 22417 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786517687384214-19665-0/minicluster-data/master-0/wal/instance:
uuid: "7eb082d4a2dd42cda806c46bcf021401"
format_stamp: "Formatted at 2026-08-12 06:55:04 on dist-test-slave-65mx"
server_key: "561cc1cf26890800513fae9e042e5eac"
server_key_iv: "d75b295fc7a9347c5e37512216ca07b9"
server_key_version: "encryptionkey@0"
I20260812 06:55:04.294893 22417 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.003s
I20260812 06:55:04.297446 22435 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 06:55:04.298826 22417 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 06:55:04.299032 22417 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786517687384214-19665-0/minicluster-data/master-0/data,/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786517687384214-19665-0/minicluster-data/master-0/wal
uuid: "7eb082d4a2dd42cda806c46bcf021401"
format_stamp: "Formatted at 2026-08-12 06:55:04 on dist-test-slave-65mx"
server_key: "561cc1cf26890800513fae9e042e5eac"
server_key_iv: "d75b295fc7a9347c5e37512216ca07b9"
server_key_version: "encryptionkey@0"
I20260812 06:55:04.299216 22417 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786517687384214-19665-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786517687384214-19665-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786517687384214-19665-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:55:04.317399 22417 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 06:55:04.320786 22417 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 06:55:04.321065 22417 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:55:04.329182 22417 rpc_server.cc:307] RPC server started. Bound to: 127.19.52.126:36197
I20260812 06:55:04.329216 22493 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.52.126:36197 every 8 connection(s)
I20260812 06:55:04.330191 22417 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786517687384214-19665-0/minicluster-data/master-0/data/info.pb
I20260812 06:55:04.333310 22494 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 06:55:04.337776 19665 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu as pid 22417
I20260812 06:55:04.337878 19665 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786517687384214-19665-0/minicluster-data/master-0/wal/instance
I20260812 06:55:04.338200 19665 external_mini_cluster.cc:1428] Setting key 7c36ebe50ca3222a7b1584b42e047486
I20260812 06:55:04.340046 22494 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7eb082d4a2dd42cda806c46bcf021401: Bootstrap starting.
I20260812 06:55:04.342528 22494 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7eb082d4a2dd42cda806c46bcf021401: Neither blocks nor log segments found. Creating new log.
I20260812 06:55:04.343318 22494 log.cc:824] T 00000000000000000000000000000000 P 7eb082d4a2dd42cda806c46bcf021401: Log is configured to *not* fsync() on all Append() calls
I20260812 06:55:04.345440 22494 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7eb082d4a2dd42cda806c46bcf021401: No bootstrap required, opened a new log
Aug 12 06:55:04 dist-test-slave-65mx krb5kdc[22404](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517704, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.19.52.126@KRBTEST.COM
I20260812 06:55:04.350324 22494 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7eb082d4a2dd42cda806c46bcf021401 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7eb082d4a2dd42cda806c46bcf021401" member_type: VOTER last_known_addr { host: "127.19.52.126" port: 36197 } }
I20260812 06:55:04.350647 22494 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7eb082d4a2dd42cda806c46bcf021401 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 06:55:04.350728 22494 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7eb082d4a2dd42cda806c46bcf021401 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7eb082d4a2dd42cda806c46bcf021401, State: Initialized, Role: FOLLOWER
I20260812 06:55:04.351255 22494 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7eb082d4a2dd42cda806c46bcf021401 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7eb082d4a2dd42cda806c46bcf021401" member_type: VOTER last_known_addr { host: "127.19.52.126" port: 36197 } }
I20260812 06:55:04.351503 22499 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7eb082d4a2dd42cda806c46bcf021401 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 06:55:04.351684 22499 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7eb082d4a2dd42cda806c46bcf021401 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7eb082d4a2dd42cda806c46bcf021401" member_type: VOTER last_known_addr { host: "127.19.52.126" port: 36197 } }
I20260812 06:55:04.351452 22494 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7eb082d4a2dd42cda806c46bcf021401 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 06:55:04.352126 22494 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7eb082d4a2dd42cda806c46bcf021401 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 06:55:04.352082 22499 leader_election.cc:302] T 00000000000000000000000000000000 P 7eb082d4a2dd42cda806c46bcf021401 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7eb082d4a2dd42cda806c46bcf021401; no voters:
I20260812 06:55:04.352244 22494 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7eb082d4a2dd42cda806c46bcf021401 [term 0 FOLLOWER]: Advancing to term 1
I20260812 06:55:04.352596 22499 leader_election.cc:288] T 00000000000000000000000000000000 P 7eb082d4a2dd42cda806c46bcf021401 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers
I20260812 06:55:04.353595 22494 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7eb082d4a2dd42cda806c46bcf021401 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7eb082d4a2dd42cda806c46bcf021401" member_type: VOTER last_known_addr { host: "127.19.52.126" port: 36197 } }
I20260812 06:55:04.353741 22494 leader_election.cc:302] T 00000000000000000000000000000000 P 7eb082d4a2dd42cda806c46bcf021401 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7eb082d4a2dd42cda806c46bcf021401; no voters:
I20260812 06:55:04.353806 22500 raft_consensus.cc:2741] T 00000000000000000000000000000000 P 7eb082d4a2dd42cda806c46bcf021401 [term 1 FOLLOWER]: Leader pre-election decision vote started in defunct term 0: won
I20260812 06:55:04.353849 22494 leader_election.cc:288] T 00000000000000000000000000000000 P 7eb082d4a2dd42cda806c46bcf021401 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 06:55:04.354171 22499 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7eb082d4a2dd42cda806c46bcf021401 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 06:55:04.354442 22499 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7eb082d4a2dd42cda806c46bcf021401 [term 1 LEADER]: Becoming Leader. State: Replica: 7eb082d4a2dd42cda806c46bcf021401, State: Running, Role: LEADER
I20260812 06:55:04.354722 22497 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:04.339900 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58158 (local address 127.19.52.126:36197)
0812 06:55:04.340433 (+ 533us) server_negotiation.cc:206] Beginning negotiation
0812 06:55:04.340453 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0812 06:55:04.340490 (+ 37us) server_negotiation.cc:407] Connection header received
0812 06:55:04.341259 (+ 769us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:55:04.341277 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:55:04.341647 (+ 370us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:55:04.342062 (+ 415us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:55:04.342901 (+ 839us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:04.343988 (+ 1087us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:04.344953 (+ 965us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:04.345444 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:04.345626 (+ 182us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:55:04.348626 (+ 3000us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:55:04.348636 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:55:04.348661 (+ 25us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:55:04.348708 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:55:04.350809 (+ 2101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:55:04.351372 (+ 563us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:55:04.351379 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:55:04.351388 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:55:04.351538 (+ 150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:55:04.351904 (+ 366us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:55:04.351906 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:55:04.351907 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:55:04.352485 (+ 578us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:55:04.352726 (+ 241us) server_negotiation.cc:1035] Waiting for connection context
0812 06:55:04.353154 (+ 428us) server_negotiation.cc:299] Negotiation successful
0812 06:55:04.353517 (+ 363us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":383,"thread_start_us":160,"threads_started":1}
I20260812 06:55:04.354833 22499 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7eb082d4a2dd42cda806c46bcf021401 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7eb082d4a2dd42cda806c46bcf021401" member_type: VOTER last_known_addr { host: "127.19.52.126" port: 36197 } }
I20260812 06:55:04.355289 22494 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7eb082d4a2dd42cda806c46bcf021401 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 06:55:04.356150 22499 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7eb082d4a2dd42cda806c46bcf021401 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7eb082d4a2dd42cda806c46bcf021401. Latest consensus state: current_term: 1 leader_uuid: "7eb082d4a2dd42cda806c46bcf021401" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7eb082d4a2dd42cda806c46bcf021401" member_type: VOTER last_known_addr { host: "127.19.52.126" port: 36197 } } }
I20260812 06:55:04.356287 22499 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7eb082d4a2dd42cda806c46bcf021401 [sys.catalog]: This master's current role is: LEADER
I20260812 06:55:04.357466 22513 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 06:55:04.357812 22500 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7eb082d4a2dd42cda806c46bcf021401 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7eb082d4a2dd42cda806c46bcf021401" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7eb082d4a2dd42cda806c46bcf021401" member_type: VOTER last_known_addr { host: "127.19.52.126" port: 36197 } } }
I20260812 06:55:04.358132 22500 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7eb082d4a2dd42cda806c46bcf021401 [sys.catalog]: This master's current role is: LEADER
I20260812 06:55:04.360426 22513 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 06:55:04.366803 22513 catalog_manager.cc:1347] Generated new cluster ID: f1ddb0cce0b44f08b8f2d207d95c27bb
I20260812 06:55:04.366899 22513 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 06:55:04.372856 22513 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 06:55:04.374150 22513 catalog_manager.cc:1504] Loading token signing keys...
I20260812 06:55:04.383895 22513 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7eb082d4a2dd42cda806c46bcf021401: Generated new TSK 0
I20260812 06:55:04.384804 22513 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.19.52.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.52.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.52.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786517687384214-19665-0/krb5kdc/kudu_127.19.52.65.keytab.
Entry for principal kudu/127.19.52.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786517687384214-19665-0/krb5kdc/kudu_127.19.52.65.keytab.
I20260812 06:55:04.424780 19665 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu
/tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786517687384214-19665-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786517687384214-19665-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786517687384214-19665-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786517687384214-19665-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.52.65:0
--local_ip_for_outbound_sockets=127.19.52.65
--webserver_interface=127.19.52.65
--webserver_port=0
--tserver_master_addrs=127.19.52.126:36197
--keytab_file=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786517687384214-19665-0/krb5kdc/kudu_127.19.52.65.keytab
--principal=kudu/127.19.52.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.52.84:43891
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786517687384214-19665-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786517687384214-19665-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786517687384214-19665-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 06:55:04.554347 22529 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:55:04.554661 22529 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:55:04.554708 22529 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 06:55:04.559415 22529 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 06:55:04.559491 22529 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 06:55:04.559573 22529 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.52.65
I20260812 06:55:04.565191 22529 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.52.84:43891
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786517687384214-19665-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786517687384214-19665-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786517687384214-19665-0/krb5kdc/kudu_127.19.52.65.keytab
--principal=kudu/127.19.52.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.52.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786517687384214-19665-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.19.52.65
--webserver_port=0
--tserver_master_addrs=127.19.52.126:36197
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22529
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.52.65
--log_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786517687384214-19665-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 06:15:54 UTC on ccycloud.quasar-rnfzce.root.comops.site
build id 4581989
I20260812 06:55:04.566354 22529 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:55:04.567430 22529 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 06:55:04.577410 22539 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:55:04.577430 22536 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:55:04.577440 22535 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:55:04.577761 22529 server_base.cc:1034] running on GCE node
I20260812 06:55:04.578222 22529 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:55:04.578913 22529 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:55:04.580104 22529 hybrid_clock.cc:648] HybridClock initialized: now 1786517704580084 us; error 30 us; skew 500 ppm
Aug 12 06:55:04 dist-test-slave-65mx krb5kdc[22404](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517704, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.52.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 06:55:04.583285 22529 init.cc:377] Logged in from keytab as kudu/127.19.52.65@KRBTEST.COM (short username kudu)
I20260812 06:55:04.584476 22529 webserver.cc:460] Webserver started at http://127.19.52.65:33689/ using document root <none> and password file <none>
I20260812 06:55:04.585158 22529 fs_manager.cc:359] Metadata directory not provided
I20260812 06:55:04.585264 22529 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:55:04.585536 22529 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 06:55:04.587908 22529 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786517687384214-19665-0/minicluster-data/ts-0/data/instance:
uuid: "e7e038c433b547afbfc1c0fbc69eab97"
format_stamp: "Formatted at 2026-08-12 06:55:04 on dist-test-slave-65mx"
server_key: "9db79419fdbe1354ad61a5c2187f045e"
server_key_iv: "23fa8a12b5c505fd05a143f548344c1e"
server_key_version: "encryptionkey@0"
I20260812 06:55:04.588627 22529 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786517687384214-19665-0/minicluster-data/ts-0/wal/instance:
uuid: "e7e038c433b547afbfc1c0fbc69eab97"
format_stamp: "Formatted at 2026-08-12 06:55:04 on dist-test-slave-65mx"
server_key: "9db79419fdbe1354ad61a5c2187f045e"
server_key_iv: "23fa8a12b5c505fd05a143f548344c1e"
server_key_version: "encryptionkey@0"
I20260812 06:55:04.592897 22529 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260812 06:55:04.595664 22548 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 06:55:04.597213 22529 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 06:55:04.597358 22529 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786517687384214-19665-0/minicluster-data/ts-0/data,/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786517687384214-19665-0/minicluster-data/ts-0/wal
uuid: "e7e038c433b547afbfc1c0fbc69eab97"
format_stamp: "Formatted at 2026-08-12 06:55:04 on dist-test-slave-65mx"
server_key: "9db79419fdbe1354ad61a5c2187f045e"
server_key_iv: "23fa8a12b5c505fd05a143f548344c1e"
server_key_version: "encryptionkey@0"
I20260812 06:55:04.597492 22529 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786517687384214-19665-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786517687384214-19665-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786517687384214-19665-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:55:04.617584 22529 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 06:55:04.621201 22529 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 06:55:04.621467 22529 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:55:04.622153 22529 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 06:55:04.623164 22529 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 06:55:04.623245 22529 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 06:55:04.623320 22529 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 06:55:04.623363 22529 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 06:55:04.633919 22529 rpc_server.cc:307] RPC server started. Bound to: 127.19.52.65:42365
I20260812 06:55:04.634081 22676 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.52.65:42365 every 8 connection(s)
I20260812 06:55:04.635016 22529 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786517687384214-19665-0/minicluster-data/ts-0/data/info.pb
I20260812 06:55:04.643873 19665 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu as pid 22529
I20260812 06:55:04.643983 19665 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786517687384214-19665-0/minicluster-data/ts-0/wal/instance
I20260812 06:55:04.644272 19665 external_mini_cluster.cc:1428] Setting key b79dbe33d794397e874b8fe832552e74
Aug 12 06:55:04 dist-test-slave-65mx krb5kdc[22404](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517704, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.52.65@KRBTEST.COM for kudu/127.19.52.126@KRBTEST.COM
I20260812 06:55:04.651041 22497 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:04.637217 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.52.65:34755 (local address 127.19.52.126:36197)
0812 06:55:04.637378 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0812 06:55:04.637382 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 06:55:04.638613 (+ 1231us) server_negotiation.cc:407] Connection header received
0812 06:55:04.639762 (+ 1149us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:55:04.639766 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:55:04.639832 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:55:04.639948 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:55:04.641346 (+ 1398us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:04.641953 (+ 607us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:04.643294 (+ 1341us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:04.643592 (+ 298us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:04.643708 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:55:04.646958 (+ 3250us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:55:04.646968 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:55:04.646971 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:55:04.646997 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:55:04.648708 (+ 1711us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:55:04.649302 (+ 594us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:55:04.649308 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:55:04.649310 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:55:04.649402 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:55:04.649786 (+ 384us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:55:04.649792 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:55:04.649794 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:55:04.649999 (+ 205us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:55:04.650120 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0812 06:55:04.650704 (+ 584us) server_negotiation.cc:299] Negotiation successful
0812 06:55:04.650855 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260812 06:55:04.651973 22680 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:04.637552 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.126:36197 (local address 127.19.52.65:34755)
0812 06:55:04.638404 (+ 852us) negotiation.cc:107] Waiting for socket to connect
0812 06:55:04.638457 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0812 06:55:04.639503 (+ 1046us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:55:04.640154 (+ 651us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:55:04.640163 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:55:04.640624 (+ 461us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:55:04.641176 (+ 552us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:55:04.641191 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:55:04.642106 (+ 915us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:55:04.642110 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:55:04.643140 (+ 1030us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:55:04.643151 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:55:04.643783 (+ 632us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:55:04.643788 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:55:04.644023 (+ 235us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:55:04.644794 (+ 771us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 06:55:04.644820 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:55:04.646763 (+ 1943us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 06:55:04.648842 (+ 2079us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:55:04.648849 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:55:04.648869 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:55:04.649186 (+ 317us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:55:04.649501 (+ 315us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:55:04.649506 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:55:04.649508 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:55:04.649680 (+ 172us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:55:04.650133 (+ 453us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 06:55:04.650142 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 06:55:04.650432 (+ 290us) client_negotiation.cc:769] Sending connection context
0812 06:55:04.650660 (+ 228us) client_negotiation.cc:240] Negotiation successful
0812 06:55:04.651043 (+ 383us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":631,"thread_start_us":140,"threads_started":1}
I20260812 06:55:04.653316 22677 heartbeater.cc:347] Connected to a master server at 127.19.52.126:36197
I20260812 06:55:04.653607 22677 heartbeater.cc:464] Registering TS with master...
I20260812 06:55:04.654249 22677 heartbeater.cc:511] Master 127.19.52.126:36197 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.19.52.66@KRBTEST.COM; defaulting to no policy
I20260812 06:55:04.656210 22453 ts_manager.cc:194] Registered new tserver with Master: e7e038c433b547afbfc1c0fbc69eab97 (127.19.52.65:42365)
I20260812 06:55:04.657737 22453 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.52.65@KRBTEST.COM'} at 127.19.52.65:34755
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.52.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.52.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786517687384214-19665-0/krb5kdc/kudu_127.19.52.66.keytab.
Entry for principal kudu/127.19.52.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786517687384214-19665-0/krb5kdc/kudu_127.19.52.66.keytab.
I20260812 06:55:04.676358 19665 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu
/tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786517687384214-19665-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786517687384214-19665-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786517687384214-19665-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786517687384214-19665-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.52.66:0
--local_ip_for_outbound_sockets=127.19.52.66
--webserver_interface=127.19.52.66
--webserver_port=0
--tserver_master_addrs=127.19.52.126:36197
--keytab_file=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786517687384214-19665-0/krb5kdc/kudu_127.19.52.66.keytab
--principal=kudu/127.19.52.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.52.84:43891
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786517687384214-19665-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786517687384214-19665-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786517687384214-19665-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 06:55:04.798458 22684 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:55:04.798768 22684 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:55:04.798815 22684 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 06:55:04.803134 22684 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 06:55:04.803212 22684 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 06:55:04.803277 22684 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.52.66
I20260812 06:55:04.808954 22684 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.52.84:43891
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786517687384214-19665-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786517687384214-19665-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786517687384214-19665-0/krb5kdc/kudu_127.19.52.66.keytab
--principal=kudu/127.19.52.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.52.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786517687384214-19665-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.19.52.66
--webserver_port=0
--tserver_master_addrs=127.19.52.126:36197
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22684
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.52.66
--log_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786517687384214-19665-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 06:15:54 UTC on ccycloud.quasar-rnfzce.root.comops.site
build id 4581989
I20260812 06:55:04.810071 22684 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:55:04.811144 22684 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 06:55:04.821498 22692 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:55:04.821478 22694 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:55:04.821542 22684 server_base.cc:1034] running on GCE node
W20260812 06:55:04.821486 22690 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:55:04.822125 22684 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:55:04.822854 22684 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:55:04.824062 22684 hybrid_clock.cc:648] HybridClock initialized: now 1786517704824028 us; error 39 us; skew 500 ppm
Aug 12 06:55:04 dist-test-slave-65mx krb5kdc[22404](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517704, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.52.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 06:55:04.827169 22684 init.cc:377] Logged in from keytab as kudu/127.19.52.66@KRBTEST.COM (short username kudu)
I20260812 06:55:04.828133 22684 webserver.cc:460] Webserver started at http://127.19.52.66:36385/ using document root <none> and password file <none>
I20260812 06:55:04.828797 22684 fs_manager.cc:359] Metadata directory not provided
I20260812 06:55:04.828892 22684 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:55:04.829128 22684 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 06:55:04.831259 22684 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786517687384214-19665-0/minicluster-data/ts-1/data/instance:
uuid: "00ee621c51b1466f822567c546e2005d"
format_stamp: "Formatted at 2026-08-12 06:55:04 on dist-test-slave-65mx"
server_key: "bcb051a4f38dff9a856d2a0ce6a44151"
server_key_iv: "78e902dddf366b217c93174b2bd3b20e"
server_key_version: "encryptionkey@0"
I20260812 06:55:04.831854 22684 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786517687384214-19665-0/minicluster-data/ts-1/wal/instance:
uuid: "00ee621c51b1466f822567c546e2005d"
format_stamp: "Formatted at 2026-08-12 06:55:04 on dist-test-slave-65mx"
server_key: "bcb051a4f38dff9a856d2a0ce6a44151"
server_key_iv: "78e902dddf366b217c93174b2bd3b20e"
server_key_version: "encryptionkey@0"
I20260812 06:55:04.835726 22684 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.002s
I20260812 06:55:04.838290 22701 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 06:55:04.839655 22684 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260812 06:55:04.839818 22684 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786517687384214-19665-0/minicluster-data/ts-1/data,/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786517687384214-19665-0/minicluster-data/ts-1/wal
uuid: "00ee621c51b1466f822567c546e2005d"
format_stamp: "Formatted at 2026-08-12 06:55:04 on dist-test-slave-65mx"
server_key: "bcb051a4f38dff9a856d2a0ce6a44151"
server_key_iv: "78e902dddf366b217c93174b2bd3b20e"
server_key_version: "encryptionkey@0"
I20260812 06:55:04.839951 22684 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786517687384214-19665-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786517687384214-19665-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786517687384214-19665-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:55:04.892980 22684 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 06:55:04.897281 22684 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 06:55:04.897639 22684 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:55:04.898378 22684 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 06:55:04.899665 22684 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 06:55:04.899760 22684 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 06:55:04.899837 22684 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 06:55:04.899884 22684 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 06:55:04.910478 22684 rpc_server.cc:307] RPC server started. Bound to: 127.19.52.66:36987
I20260812 06:55:04.910498 22830 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.52.66:36987 every 8 connection(s)
I20260812 06:55:04.911522 22684 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786517687384214-19665-0/minicluster-data/ts-1/data/info.pb
I20260812 06:55:04.915665 19665 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu as pid 22684
I20260812 06:55:04.915779 19665 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786517687384214-19665-0/minicluster-data/ts-1/wal/instance
I20260812 06:55:04.916065 19665 external_mini_cluster.cc:1428] Setting key 969a7b8ed9a7d5b0af470026cc8e6b7b
Aug 12 06:55:04 dist-test-slave-65mx krb5kdc[22404](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517704, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.52.66@KRBTEST.COM for kudu/127.19.52.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.19.52.67@KRBTEST.COM; defaulting to no policy
I20260812 06:55:04.928305 22497 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:04.913771 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.52.66:55477 (local address 127.19.52.126:36197)
0812 06:55:04.913954 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0812 06:55:04.913959 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 06:55:04.915057 (+ 1098us) server_negotiation.cc:407] Connection header received
0812 06:55:04.916218 (+ 1161us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:55:04.916234 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:55:04.916295 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:55:04.916385 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:55:04.917879 (+ 1494us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:04.918424 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:04.920155 (+ 1731us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:04.920671 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:04.920829 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:55:04.923840 (+ 3011us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:55:04.923844 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:55:04.923856 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:55:04.923882 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:55:04.925595 (+ 1713us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:55:04.926358 (+ 763us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:55:04.926361 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:55:04.926362 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:55:04.926456 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:55:04.926905 (+ 449us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:55:04.926907 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:55:04.926908 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:55:04.927136 (+ 228us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:55:04.927215 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0812 06:55:04.927970 (+ 755us) server_negotiation.cc:299] Negotiation successful
0812 06:55:04.928174 (+ 204us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":85}
I20260812 06:55:04.929365 22833 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:04.914215 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.126:36197 (local address 127.19.52.66:55477)
0812 06:55:04.914911 (+ 696us) negotiation.cc:107] Waiting for socket to connect
0812 06:55:04.914965 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0812 06:55:04.915966 (+ 1001us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:55:04.916641 (+ 675us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:55:04.916648 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:55:04.917108 (+ 460us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:55:04.917733 (+ 625us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:55:04.917752 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:55:04.918562 (+ 810us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:55:04.918568 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:55:04.920022 (+ 1454us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:55:04.920032 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:55:04.920857 (+ 825us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:55:04.920860 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:55:04.921052 (+ 192us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:55:04.921689 (+ 637us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 06:55:04.921706 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:55:04.923600 (+ 1894us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 06:55:04.925737 (+ 2137us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:55:04.925746 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:55:04.925768 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:55:04.926236 (+ 468us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:55:04.926564 (+ 328us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:55:04.926570 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:55:04.926575 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:55:04.926804 (+ 229us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:55:04.927262 (+ 458us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 06:55:04.927274 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 06:55:04.927699 (+ 425us) client_negotiation.cc:769] Sending connection context
0812 06:55:04.927947 (+ 248us) client_negotiation.cc:240] Negotiation successful
0812 06:55:04.928383 (+ 436us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":501,"thread_start_us":136,"threads_started":1}
I20260812 06:55:04.930666 22831 heartbeater.cc:347] Connected to a master server at 127.19.52.126:36197
I20260812 06:55:04.930935 22831 heartbeater.cc:464] Registering TS with master...
I20260812 06:55:04.931528 22831 heartbeater.cc:511] Master 127.19.52.126:36197 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.52.67@KRBTEST.COM" created.
I20260812 06:55:04.932767 22453 ts_manager.cc:194] Registered new tserver with Master: 00ee621c51b1466f822567c546e2005d (127.19.52.66:36987)
I20260812 06:55:04.933434 22453 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.52.66@KRBTEST.COM'} at 127.19.52.66:55477
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.52.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786517687384214-19665-0/krb5kdc/kudu_127.19.52.67.keytab.
Entry for principal kudu/127.19.52.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786517687384214-19665-0/krb5kdc/kudu_127.19.52.67.keytab.
I20260812 06:55:04.952200 19665 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu
/tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786517687384214-19665-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786517687384214-19665-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786517687384214-19665-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786517687384214-19665-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.52.67:0
--local_ip_for_outbound_sockets=127.19.52.67
--webserver_interface=127.19.52.67
--webserver_port=0
--tserver_master_addrs=127.19.52.126:36197
--keytab_file=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786517687384214-19665-0/krb5kdc/kudu_127.19.52.67.keytab
--principal=kudu/127.19.52.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.52.84:43891
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786517687384214-19665-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786517687384214-19665-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786517687384214-19665-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 06:55:05.072381 22836 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:55:05.072736 22836 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:55:05.072829 22836 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 06:55:05.077209 22836 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 06:55:05.077279 22836 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 06:55:05.077364 22836 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.52.67
I20260812 06:55:05.082934 22836 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.52.84:43891
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786517687384214-19665-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786517687384214-19665-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786517687384214-19665-0/krb5kdc/kudu_127.19.52.67.keytab
--principal=kudu/127.19.52.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.52.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786517687384214-19665-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.19.52.67
--webserver_port=0
--tserver_master_addrs=127.19.52.126:36197
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22836
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.52.67
--log_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786517687384214-19665-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 06:15:54 UTC on ccycloud.quasar-rnfzce.root.comops.site
build id 4581989
I20260812 06:55:05.084103 22836 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:55:05.085197 22836 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 06:55:05.095021 22845 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:55:05.095023 22848 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:55:05.095340 22846 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:55:05.095321 22836 server_base.cc:1034] running on GCE node
I20260812 06:55:05.095849 22836 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:55:05.096534 22836 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:55:05.097747 22836 hybrid_clock.cc:648] HybridClock initialized: now 1786517705097727 us; error 34 us; skew 500 ppm
Aug 12 06:55:05 dist-test-slave-65mx krb5kdc[22404](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517705, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.52.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 06:55:05.101054 22836 init.cc:377] Logged in from keytab as kudu/127.19.52.67@KRBTEST.COM (short username kudu)
I20260812 06:55:05.102051 22836 webserver.cc:460] Webserver started at http://127.19.52.67:33355/ using document root <none> and password file <none>
I20260812 06:55:05.102602 22836 fs_manager.cc:359] Metadata directory not provided
I20260812 06:55:05.102664 22836 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:55:05.102836 22836 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 06:55:05.105088 22836 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786517687384214-19665-0/minicluster-data/ts-2/data/instance:
uuid: "40df855da7384909badb87a60369386f"
format_stamp: "Formatted at 2026-08-12 06:55:05 on dist-test-slave-65mx"
server_key: "ff6c3695c2e2a15524da56457645d6b5"
server_key_iv: "7f7d42daa62acb331347e70da31d59fb"
server_key_version: "encryptionkey@0"
I20260812 06:55:05.105621 22836 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786517687384214-19665-0/minicluster-data/ts-2/wal/instance:
uuid: "40df855da7384909badb87a60369386f"
format_stamp: "Formatted at 2026-08-12 06:55:05 on dist-test-slave-65mx"
server_key: "ff6c3695c2e2a15524da56457645d6b5"
server_key_iv: "7f7d42daa62acb331347e70da31d59fb"
server_key_version: "encryptionkey@0"
I20260812 06:55:05.109500 22836 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260812 06:55:05.112263 22859 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 06:55:05.113675 22836 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260812 06:55:05.113791 22836 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786517687384214-19665-0/minicluster-data/ts-2/data,/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786517687384214-19665-0/minicluster-data/ts-2/wal
uuid: "40df855da7384909badb87a60369386f"
format_stamp: "Formatted at 2026-08-12 06:55:05 on dist-test-slave-65mx"
server_key: "ff6c3695c2e2a15524da56457645d6b5"
server_key_iv: "7f7d42daa62acb331347e70da31d59fb"
server_key_version: "encryptionkey@0"
I20260812 06:55:05.113931 22836 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786517687384214-19665-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786517687384214-19665-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786517687384214-19665-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:55:05.139091 22836 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 06:55:05.142450 22836 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 06:55:05.142679 22836 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:55:05.143313 22836 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 06:55:05.144258 22836 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 06:55:05.144308 22836 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 06:55:05.144372 22836 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 06:55:05.144412 22836 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 06:55:05.154281 22836 rpc_server.cc:307] RPC server started. Bound to: 127.19.52.67:36817
I20260812 06:55:05.154325 22987 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.52.67:36817 every 8 connection(s)
I20260812 06:55:05.155347 22836 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786517687384214-19665-0/minicluster-data/ts-2/data/info.pb
I20260812 06:55:05.160825 19665 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu as pid 22836
I20260812 06:55:05.160969 19665 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786517687384214-19665-0/minicluster-data/ts-2/wal/instance
I20260812 06:55:05.161289 19665 external_mini_cluster.cc:1428] Setting key d5461cbfe8c88b7f0ef07c6f5c6ffc9f
Aug 12 06:55:05 dist-test-slave-65mx krb5kdc[22404](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517705, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.52.67@KRBTEST.COM for kudu/127.19.52.126@KRBTEST.COM
I20260812 06:55:05.170681 22497 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:05.157473 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.52.67:55849 (local address 127.19.52.126:36197)
0812 06:55:05.157625 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0812 06:55:05.157629 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 06:55:05.158789 (+ 1160us) server_negotiation.cc:407] Connection header received
0812 06:55:05.159687 (+ 898us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:55:05.159691 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:55:05.159745 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:55:05.159830 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:55:05.161196 (+ 1366us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:05.161775 (+ 579us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:05.163316 (+ 1541us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:05.163605 (+ 289us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:05.163690 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:55:05.166614 (+ 2924us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:55:05.166620 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:55:05.166626 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:55:05.166657 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:55:05.168353 (+ 1696us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:55:05.169111 (+ 758us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:55:05.169114 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:55:05.169115 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:55:05.169202 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:55:05.169564 (+ 362us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:55:05.169566 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:55:05.169567 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:55:05.169753 (+ 186us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:55:05.169833 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0812 06:55:05.170418 (+ 585us) server_negotiation.cc:299] Negotiation successful
0812 06:55:05.170555 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260812 06:55:05.171605 22992 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:05.157916 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.126:36197 (local address 127.19.52.67:55849)
0812 06:55:05.158628 (+ 712us) negotiation.cc:107] Waiting for socket to connect
0812 06:55:05.158670 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0812 06:55:05.159460 (+ 790us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:55:05.160009 (+ 549us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:55:05.160016 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:55:05.160388 (+ 372us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:55:05.161029 (+ 641us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:55:05.161052 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:55:05.161912 (+ 860us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:55:05.161916 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:55:05.163154 (+ 1238us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:55:05.163165 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:55:05.163745 (+ 580us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:55:05.163748 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:55:05.163889 (+ 141us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:55:05.164464 (+ 575us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 06:55:05.164481 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:55:05.166369 (+ 1888us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 06:55:05.168673 (+ 2304us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:55:05.168678 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:55:05.168692 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:55:05.169002 (+ 310us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:55:05.169295 (+ 293us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:55:05.169297 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:55:05.169298 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:55:05.169444 (+ 146us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:55:05.169873 (+ 429us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 06:55:05.169879 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 06:55:05.170156 (+ 277us) client_negotiation.cc:769] Sending connection context
0812 06:55:05.170325 (+ 169us) client_negotiation.cc:240] Negotiation successful
0812 06:55:05.170614 (+ 289us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":515,"thread_start_us":124,"threads_started":1}
I20260812 06:55:05.172773 22988 heartbeater.cc:347] Connected to a master server at 127.19.52.126:36197
I20260812 06:55:05.173079 22988 heartbeater.cc:464] Registering TS with master...
I20260812 06:55:05.173591 22988 heartbeater.cc:511] Master 127.19.52.126:36197 requested a full tablet report, sending...
I20260812 06:55:05.174762 22453 ts_manager.cc:194] Registered new tserver with Master: 40df855da7384909badb87a60369386f (127.19.52.67:36817)
I20260812 06:55:05.175499 22453 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.52.67@KRBTEST.COM'} at 127.19.52.67:55849
I20260812 06:55:05.175721 19665 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260812 06:55:05.191730 22497 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 06:55:05.190122 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58172 (local address 127.19.52.126:36197)
0812 06:55:05.190266 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0812 06:55:05.190270 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 06:55:05.190391 (+ 121us) server_negotiation.cc:407] Connection header received
0812 06:55:05.190587 (+ 196us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:55:05.190591 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:55:05.190643 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:55:05.190730 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:55:05.191616 (+ 886us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:58172: BlockingRecv error: recv got EOF from 127.0.0.1:58172 (error 108)
Metrics: {"server-negotiator.queue_time_us":53}
I20260812 06:55:05.193300 19665 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu with pid 22529
I20260812 06:55:05.206720 19665 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu with pid 22684
I20260812 06:55:05.217869 19665 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu with pid 22836
I20260812 06:55:05.228083 19665 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu with pid 22417
2026-08-12T06:55:05Z chronyd exiting
[ OK ] SecurityITest.TestNoKerberosCredentials (2414 ms)
[ RUN ] SecurityITest.SaslPlainFallback
Loading random data
Initializing database '/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786517687384214-19665-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 06:55:05 dist-test-slave-65mx krb5kdc[23005](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 06:55:05 dist-test-slave-65mx krb5kdc[23005](info): set up 2 sockets
Aug 12 06:55:05 dist-test-slave-65mx krb5kdc[23005](info): commencing operation
krb5kdc: starting...
W20260812 06:55:06.364059 19665 mini_kdc.cc:121] Time spent starting KDC: real 1.111s user 0.008s sys 0.000s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786517687384214-19665-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786517687384214-19665-0/krb5kdc/test-user.keytab.
Aug 12 06:55:06 dist-test-slave-65mx krb5kdc[23005](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517706, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T06:55:06Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T06:55:06Z Disabled control of system clock
WARNING: no policy specified for kudu/127.19.52.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.52.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.52.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786517687384214-19665-0/krb5kdc/kudu_127.19.52.126.keytab.
Entry for principal kudu/127.19.52.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786517687384214-19665-0/krb5kdc/kudu_127.19.52.126.keytab.
I20260812 06:55:06.500363 19665 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu
/tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786517687384214-19665-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786517687384214-19665-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786517687384214-19665-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786517687384214-19665-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.52.126:45877
--webserver_interface=127.19.52.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786517687384214-19665-0/krb5kdc/kudu_127.19.52.126.keytab
--principal=kudu/127.19.52.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.52.84:33519
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.19.52.126:45877
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc-authentication=optional
--user-acl=* with env {KRB5CCNAME=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786517687384214-19665-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786517687384214-19665-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786517687384214-19665-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 06:55:06.614718 23023 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:55:06.615057 23023 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:55:06.615139 23023 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 06:55:06.619293 23023 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 06:55:06.619386 23023 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 06:55:06.619421 23023 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 06:55:06.619465 23023 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 06:55:06.619501 23023 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 06:55:06.624982 23023 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.52.84:33519
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786517687384214-19665-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786517687384214-19665-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.19.52.126:45877
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786517687384214-19665-0/krb5kdc/kudu_127.19.52.126.keytab
--principal=kudu/127.19.52.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.52.126:45877
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786517687384214-19665-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.19.52.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23023
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786517687384214-19665-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 06:15:54 UTC on ccycloud.quasar-rnfzce.root.comops.site
build id 4581989
I20260812 06:55:06.626145 23023 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:55:06.627205 23023 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 06:55:06.636051 23029 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:55:06.636051 23033 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:55:06.636051 23028 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:55:06.636673 23023 server_base.cc:1034] running on GCE node
I20260812 06:55:06.637310 23023 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:55:06.638636 23023 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:55:06.639840 23023 hybrid_clock.cc:648] HybridClock initialized: now 1786517706639828 us; error 30 us; skew 500 ppm
Aug 12 06:55:06 dist-test-slave-65mx krb5kdc[23005](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517706, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.52.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 06:55:06.642958 23023 init.cc:377] Logged in from keytab as kudu/127.19.52.126@KRBTEST.COM (short username kudu)
I20260812 06:55:06.644019 23023 webserver.cc:460] Webserver started at http://127.19.52.126:40549/ using document root <none> and password file <none>
I20260812 06:55:06.644753 23023 fs_manager.cc:359] Metadata directory not provided
I20260812 06:55:06.644848 23023 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:55:06.645094 23023 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 06:55:06.647243 23023 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786517687384214-19665-0/minicluster-data/master-0/data/instance:
uuid: "ef76fa1cfbc34718bfe2aa4c71f23e5d"
format_stamp: "Formatted at 2026-08-12 06:55:06 on dist-test-slave-65mx"
server_key: "535ee58818b655263ba6730509830cfd"
server_key_iv: "4444049d9a643c1dbafd3dd50d7b8862"
server_key_version: "encryptionkey@0"
I20260812 06:55:06.647846 23023 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786517687384214-19665-0/minicluster-data/master-0/wal/instance:
uuid: "ef76fa1cfbc34718bfe2aa4c71f23e5d"
format_stamp: "Formatted at 2026-08-12 06:55:06 on dist-test-slave-65mx"
server_key: "535ee58818b655263ba6730509830cfd"
server_key_iv: "4444049d9a643c1dbafd3dd50d7b8862"
server_key_version: "encryptionkey@0"
I20260812 06:55:06.651894 23023 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260812 06:55:06.654595 23040 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 06:55:06.655859 23023 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260812 06:55:06.656006 23023 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786517687384214-19665-0/minicluster-data/master-0/data,/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786517687384214-19665-0/minicluster-data/master-0/wal
uuid: "ef76fa1cfbc34718bfe2aa4c71f23e5d"
format_stamp: "Formatted at 2026-08-12 06:55:06 on dist-test-slave-65mx"
server_key: "535ee58818b655263ba6730509830cfd"
server_key_iv: "4444049d9a643c1dbafd3dd50d7b8862"
server_key_version: "encryptionkey@0"
I20260812 06:55:06.656165 23023 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786517687384214-19665-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786517687384214-19665-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786517687384214-19665-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:55:06.685353 23023 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 06:55:06.688513 23023 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 06:55:06.688761 23023 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:55:06.696357 23023 rpc_server.cc:307] RPC server started. Bound to: 127.19.52.126:45877
I20260812 06:55:06.696825 23099 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.52.126:45877 every 8 connection(s)
I20260812 06:55:06.697445 23023 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786517687384214-19665-0/minicluster-data/master-0/data/info.pb
I20260812 06:55:06.698246 19665 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu as pid 23023
I20260812 06:55:06.698378 19665 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786517687384214-19665-0/minicluster-data/master-0/wal/instance
I20260812 06:55:06.698691 19665 external_mini_cluster.cc:1428] Setting key 7974cfa2329c7f0c118c592f23a926d7
I20260812 06:55:06.701090 23100 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 06:55:06.707402 23100 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ef76fa1cfbc34718bfe2aa4c71f23e5d: Bootstrap starting.
Aug 12 06:55:06 dist-test-slave-65mx krb5kdc[23005](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517706, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.19.52.126@KRBTEST.COM
I20260812 06:55:06.710258 23100 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ef76fa1cfbc34718bfe2aa4c71f23e5d: Neither blocks nor log segments found. Creating new log.
I20260812 06:55:06.711057 23100 log.cc:824] T 00000000000000000000000000000000 P ef76fa1cfbc34718bfe2aa4c71f23e5d: Log is configured to *not* fsync() on all Append() calls
I20260812 06:55:06.713322 23100 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ef76fa1cfbc34718bfe2aa4c71f23e5d: No bootstrap required, opened a new log
I20260812 06:55:06.713521 23103 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:06.700118 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52664 (local address 127.19.52.126:45877)
0812 06:55:06.700634 (+ 516us) server_negotiation.cc:206] Beginning negotiation
0812 06:55:06.700650 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0812 06:55:06.700685 (+ 35us) server_negotiation.cc:407] Connection header received
0812 06:55:06.701363 (+ 678us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:55:06.701380 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:55:06.701697 (+ 317us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:55:06.702018 (+ 321us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:55:06.702861 (+ 843us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:06.703855 (+ 994us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:06.704788 (+ 933us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:06.705261 (+ 473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:06.705432 (+ 171us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:55:06.708453 (+ 3021us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:55:06.708458 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:55:06.708471 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:55:06.708499 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:55:06.710355 (+ 1856us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:55:06.710846 (+ 491us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:55:06.710856 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:55:06.710862 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:55:06.710978 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:55:06.711273 (+ 295us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:55:06.711277 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:55:06.711279 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:55:06.711835 (+ 556us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:55:06.712078 (+ 243us) server_negotiation.cc:1035] Waiting for connection context
0812 06:55:06.712376 (+ 298us) server_negotiation.cc:299] Negotiation successful
0812 06:55:06.712734 (+ 358us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":315,"thread_start_us":148,"threads_started":1}
I20260812 06:55:06.716688 19665 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 06:55:06.717928 23100 raft_consensus.cc:348] T 00000000000000000000000000000000 P ef76fa1cfbc34718bfe2aa4c71f23e5d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ef76fa1cfbc34718bfe2aa4c71f23e5d" member_type: VOTER last_known_addr { host: "127.19.52.126" port: 45877 } }
I20260812 06:55:06.718194 23100 raft_consensus.cc:374] T 00000000000000000000000000000000 P ef76fa1cfbc34718bfe2aa4c71f23e5d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 06:55:06.718279 23100 raft_consensus.cc:729] T 00000000000000000000000000000000 P ef76fa1cfbc34718bfe2aa4c71f23e5d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ef76fa1cfbc34718bfe2aa4c71f23e5d, State: Initialized, Role: FOLLOWER
I20260812 06:55:06.718713 23100 consensus_queue.cc:260] T 00000000000000000000000000000000 P ef76fa1cfbc34718bfe2aa4c71f23e5d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ef76fa1cfbc34718bfe2aa4c71f23e5d" member_type: VOTER last_known_addr { host: "127.19.52.126" port: 45877 } }
I20260812 06:55:06.718873 23100 raft_consensus.cc:388] T 00000000000000000000000000000000 P ef76fa1cfbc34718bfe2aa4c71f23e5d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 06:55:06.718964 23100 raft_consensus.cc:482] T 00000000000000000000000000000000 P ef76fa1cfbc34718bfe2aa4c71f23e5d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 06:55:06.719099 23100 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ef76fa1cfbc34718bfe2aa4c71f23e5d [term 0 FOLLOWER]: Advancing to term 1
I20260812 06:55:06.720141 23100 raft_consensus.cc:504] T 00000000000000000000000000000000 P ef76fa1cfbc34718bfe2aa4c71f23e5d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ef76fa1cfbc34718bfe2aa4c71f23e5d" member_type: VOTER last_known_addr { host: "127.19.52.126" port: 45877 } }
I20260812 06:55:06.720585 23100 leader_election.cc:302] T 00000000000000000000000000000000 P ef76fa1cfbc34718bfe2aa4c71f23e5d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ef76fa1cfbc34718bfe2aa4c71f23e5d; no voters:
I20260812 06:55:06.721004 23100 leader_election.cc:288] T 00000000000000000000000000000000 P ef76fa1cfbc34718bfe2aa4c71f23e5d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 06:55:06.721343 23106 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ef76fa1cfbc34718bfe2aa4c71f23e5d [term 1 FOLLOWER]: Leader election won for term 1
I20260812 06:55:06.721700 23106 raft_consensus.cc:686] T 00000000000000000000000000000000 P ef76fa1cfbc34718bfe2aa4c71f23e5d [term 1 LEADER]: Becoming Leader. State: Replica: ef76fa1cfbc34718bfe2aa4c71f23e5d, State: Running, Role: LEADER
I20260812 06:55:06.722088 23106 consensus_queue.cc:237] T 00000000000000000000000000000000 P ef76fa1cfbc34718bfe2aa4c71f23e5d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ef76fa1cfbc34718bfe2aa4c71f23e5d" member_type: VOTER last_known_addr { host: "127.19.52.126" port: 45877 } }
I20260812 06:55:06.722626 23100 sys_catalog.cc:564] T 00000000000000000000000000000000 P ef76fa1cfbc34718bfe2aa4c71f23e5d [sys.catalog]: configured and running, proceeding with master startup.
I20260812 06:55:06.723906 23108 sys_catalog.cc:455] T 00000000000000000000000000000000 P ef76fa1cfbc34718bfe2aa4c71f23e5d [sys.catalog]: SysCatalogTable state changed. Reason: New leader ef76fa1cfbc34718bfe2aa4c71f23e5d. Latest consensus state: current_term: 1 leader_uuid: "ef76fa1cfbc34718bfe2aa4c71f23e5d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ef76fa1cfbc34718bfe2aa4c71f23e5d" member_type: VOTER last_known_addr { host: "127.19.52.126" port: 45877 } } }
I20260812 06:55:06.724062 23108 sys_catalog.cc:458] T 00000000000000000000000000000000 P ef76fa1cfbc34718bfe2aa4c71f23e5d [sys.catalog]: This master's current role is: LEADER
I20260812 06:55:06.724483 23107 sys_catalog.cc:455] T 00000000000000000000000000000000 P ef76fa1cfbc34718bfe2aa4c71f23e5d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ef76fa1cfbc34718bfe2aa4c71f23e5d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ef76fa1cfbc34718bfe2aa4c71f23e5d" member_type: VOTER last_known_addr { host: "127.19.52.126" port: 45877 } } }
I20260812 06:55:06.724596 23116 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 06:55:06.724653 23107 sys_catalog.cc:458] T 00000000000000000000000000000000 P ef76fa1cfbc34718bfe2aa4c71f23e5d [sys.catalog]: This master's current role is: LEADER
I20260812 06:55:06.728305 23116 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 06:55:06.735270 23116 catalog_manager.cc:1347] Generated new cluster ID: 045cb7caf1bf4631ad78c4eb6896be57
I20260812 06:55:06.735432 23116 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 06:55:06.743096 23103 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:06.736886 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52674 (local address 127.19.52.126:45877)
0812 06:55:06.737025 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0812 06:55:06.737030 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 06:55:06.737317 (+ 287us) server_negotiation.cc:407] Connection header received
0812 06:55:06.737459 (+ 142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:55:06.737462 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:55:06.737517 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:55:06.737607 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:55:06.738245 (+ 638us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:06.739120 (+ 875us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:06.740332 (+ 1212us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:06.740757 (+ 425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:06.740861 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:55:06.741875 (+ 1014us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:55:06.741879 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:55:06.741882 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 06:55:06.742552 (+ 670us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:55:06.742591 (+ 39us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 06:55:06.742611 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:55:06.742707 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0812 06:55:06.742825 (+ 118us) server_negotiation.cc:299] Negotiation successful
0812 06:55:06.742890 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260812 06:55:06.748490 23116 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 06:55:06.749745 23116 catalog_manager.cc:1504] Loading token signing keys...
I20260812 06:55:06.757241 23116 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ef76fa1cfbc34718bfe2aa4c71f23e5d: Generated new TSK 0
I20260812 06:55:06.758004 23116 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 06:55:06.770709 19665 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu with pid 23023
2026-08-12T06:55:06Z chronyd exiting
[ OK ] SecurityITest.SaslPlainFallback (1541 ms)
[ RUN ] SecurityITest.TestUnauthorizedClientKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786517687384214-19665-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 06:55:06 dist-test-slave-65mx krb5kdc[23136](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 06:55:06 dist-test-slave-65mx krb5kdc[23136](info): set up 2 sockets
Aug 12 06:55:06 dist-test-slave-65mx krb5kdc[23136](info): commencing operation
krb5kdc: starting...
W20260812 06:55:07.925091 19665 mini_kdc.cc:121] Time spent starting KDC: real 1.130s user 0.004s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786517687384214-19665-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786517687384214-19665-0/krb5kdc/test-user.keytab.
Aug 12 06:55:08 dist-test-slave-65mx krb5kdc[23136](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517708, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T06:55:08Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T06:55:08Z Disabled control of system clock
WARNING: no policy specified for kudu/127.19.52.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.52.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.52.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786517687384214-19665-0/krb5kdc/kudu_127.19.52.126.keytab.
Entry for principal kudu/127.19.52.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786517687384214-19665-0/krb5kdc/kudu_127.19.52.126.keytab.
I20260812 06:55:08.064205 19665 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu
/tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786517687384214-19665-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786517687384214-19665-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786517687384214-19665-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786517687384214-19665-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.52.126:43847
--webserver_interface=127.19.52.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786517687384214-19665-0/krb5kdc/kudu_127.19.52.126.keytab
--principal=kudu/127.19.52.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.52.84:44461
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.19.52.126:43847
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786517687384214-19665-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786517687384214-19665-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786517687384214-19665-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 06:55:08.184290 23152 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:55:08.184695 23152 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:55:08.184773 23152 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 06:55:08.189074 23152 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 06:55:08.189178 23152 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 06:55:08.189213 23152 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 06:55:08.189254 23152 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 06:55:08.189296 23152 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 06:55:08.194986 23152 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.52.84:44461
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786517687384214-19665-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786517687384214-19665-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.19.52.126:43847
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786517687384214-19665-0/krb5kdc/kudu_127.19.52.126.keytab
--principal=kudu/127.19.52.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.52.126:43847
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786517687384214-19665-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.19.52.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23152
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786517687384214-19665-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 06:15:54 UTC on ccycloud.quasar-rnfzce.root.comops.site
build id 4581989
I20260812 06:55:08.196194 23152 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:55:08.197319 23152 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 06:55:08.206668 23159 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:55:08.206672 23158 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:55:08.206672 23161 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:55:08.206924 23152 server_base.cc:1034] running on GCE node
I20260812 06:55:08.207762 23152 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:55:08.209041 23152 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:55:08.210223 23152 hybrid_clock.cc:648] HybridClock initialized: now 1786517708210183 us; error 48 us; skew 500 ppm
Aug 12 06:55:08 dist-test-slave-65mx krb5kdc[23136](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517708, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.52.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 06:55:08.216306 23152 init.cc:377] Logged in from keytab as kudu/127.19.52.126@KRBTEST.COM (short username kudu)
I20260812 06:55:08.217434 23152 webserver.cc:460] Webserver started at http://127.19.52.126:38543/ using document root <none> and password file <none>
I20260812 06:55:08.218031 23152 fs_manager.cc:359] Metadata directory not provided
I20260812 06:55:08.218096 23152 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:55:08.218317 23152 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 06:55:08.220464 23152 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786517687384214-19665-0/minicluster-data/master-0/data/instance:
uuid: "6e473db0b5004d19bc5d7b0876382699"
format_stamp: "Formatted at 2026-08-12 06:55:08 on dist-test-slave-65mx"
server_key: "476639bc8df9f4230470a3ae3c7ba864"
server_key_iv: "36df18cbd592cd7198312def60c9841d"
server_key_version: "encryptionkey@0"
I20260812 06:55:08.221133 23152 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786517687384214-19665-0/minicluster-data/master-0/wal/instance:
uuid: "6e473db0b5004d19bc5d7b0876382699"
format_stamp: "Formatted at 2026-08-12 06:55:08 on dist-test-slave-65mx"
server_key: "476639bc8df9f4230470a3ae3c7ba864"
server_key_iv: "36df18cbd592cd7198312def60c9841d"
server_key_version: "encryptionkey@0"
I20260812 06:55:08.225215 23152 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260812 06:55:08.227934 23170 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 06:55:08.229362 23152 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 06:55:08.229554 23152 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786517687384214-19665-0/minicluster-data/master-0/data,/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786517687384214-19665-0/minicluster-data/master-0/wal
uuid: "6e473db0b5004d19bc5d7b0876382699"
format_stamp: "Formatted at 2026-08-12 06:55:08 on dist-test-slave-65mx"
server_key: "476639bc8df9f4230470a3ae3c7ba864"
server_key_iv: "36df18cbd592cd7198312def60c9841d"
server_key_version: "encryptionkey@0"
I20260812 06:55:08.229683 23152 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786517687384214-19665-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786517687384214-19665-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786517687384214-19665-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:55:08.248160 23152 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 06:55:08.251780 23152 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 06:55:08.252009 23152 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:55:08.259421 23152 rpc_server.cc:307] RPC server started. Bound to: 127.19.52.126:43847
I20260812 06:55:08.259722 23229 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.52.126:43847 every 8 connection(s)
I20260812 06:55:08.260430 23152 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786517687384214-19665-0/minicluster-data/master-0/data/info.pb
I20260812 06:55:08.262161 19665 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu as pid 23152
I20260812 06:55:08.262301 19665 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786517687384214-19665-0/minicluster-data/master-0/wal/instance
I20260812 06:55:08.262616 19665 external_mini_cluster.cc:1428] Setting key 6d4c1396a7d3de092e5a89841651824e
I20260812 06:55:08.263815 23231 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 06:55:08.271340 23231 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6e473db0b5004d19bc5d7b0876382699: Bootstrap starting.
Aug 12 06:55:08 dist-test-slave-65mx krb5kdc[23136](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517708, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.19.52.126@KRBTEST.COM
I20260812 06:55:08.274729 23231 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6e473db0b5004d19bc5d7b0876382699: Neither blocks nor log segments found. Creating new log.
I20260812 06:55:08.275508 23231 log.cc:824] T 00000000000000000000000000000000 P 6e473db0b5004d19bc5d7b0876382699: Log is configured to *not* fsync() on all Append() calls
I20260812 06:55:08.278120 23231 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6e473db0b5004d19bc5d7b0876382699: No bootstrap required, opened a new log
I20260812 06:55:08.279309 23235 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:08.264203 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55088 (local address 127.19.52.126:43847)
0812 06:55:08.264634 (+ 431us) server_negotiation.cc:206] Beginning negotiation
0812 06:55:08.264651 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0812 06:55:08.264907 (+ 256us) server_negotiation.cc:407] Connection header received
0812 06:55:08.265684 (+ 777us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:55:08.265701 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:55:08.266085 (+ 384us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:55:08.266375 (+ 290us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:55:08.267343 (+ 968us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:08.268360 (+ 1017us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:08.269761 (+ 1401us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:08.270276 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:08.270424 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:55:08.274034 (+ 3610us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:55:08.274040 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:55:08.274054 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:55:08.274091 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:55:08.276166 (+ 2075us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:55:08.276803 (+ 637us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:55:08.276809 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:55:08.276814 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:55:08.276996 (+ 182us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:55:08.277288 (+ 292us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:55:08.277290 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:55:08.277291 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:55:08.277701 (+ 410us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:55:08.277848 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0812 06:55:08.278202 (+ 354us) server_negotiation.cc:299] Negotiation successful
0812 06:55:08.278555 (+ 353us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":257,"thread_start_us":124,"threads_started":1}
I20260812 06:55:08.282034 23231 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6e473db0b5004d19bc5d7b0876382699 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6e473db0b5004d19bc5d7b0876382699" member_type: VOTER last_known_addr { host: "127.19.52.126" port: 43847 } }
I20260812 06:55:08.282317 23231 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6e473db0b5004d19bc5d7b0876382699 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 06:55:08.282408 23231 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6e473db0b5004d19bc5d7b0876382699 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6e473db0b5004d19bc5d7b0876382699, State: Initialized, Role: FOLLOWER
I20260812 06:55:08.282896 23231 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6e473db0b5004d19bc5d7b0876382699 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6e473db0b5004d19bc5d7b0876382699" member_type: VOTER last_known_addr { host: "127.19.52.126" port: 43847 } }
I20260812 06:55:08.283062 23231 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6e473db0b5004d19bc5d7b0876382699 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 06:55:08.283149 23231 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6e473db0b5004d19bc5d7b0876382699 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 06:55:08.283267 23231 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6e473db0b5004d19bc5d7b0876382699 [term 0 FOLLOWER]: Advancing to term 1
I20260812 06:55:08.284338 23231 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6e473db0b5004d19bc5d7b0876382699 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6e473db0b5004d19bc5d7b0876382699" member_type: VOTER last_known_addr { host: "127.19.52.126" port: 43847 } }
I20260812 06:55:08.284757 23231 leader_election.cc:302] T 00000000000000000000000000000000 P 6e473db0b5004d19bc5d7b0876382699 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6e473db0b5004d19bc5d7b0876382699; no voters:
I20260812 06:55:08.285109 23231 leader_election.cc:288] T 00000000000000000000000000000000 P 6e473db0b5004d19bc5d7b0876382699 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 06:55:08.285271 23237 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6e473db0b5004d19bc5d7b0876382699 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 06:55:08.285586 23237 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6e473db0b5004d19bc5d7b0876382699 [term 1 LEADER]: Becoming Leader. State: Replica: 6e473db0b5004d19bc5d7b0876382699, State: Running, Role: LEADER
I20260812 06:55:08.285931 23237 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6e473db0b5004d19bc5d7b0876382699 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6e473db0b5004d19bc5d7b0876382699" member_type: VOTER last_known_addr { host: "127.19.52.126" port: 43847 } }
I20260812 06:55:08.286440 23231 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6e473db0b5004d19bc5d7b0876382699 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 06:55:08.287776 23239 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6e473db0b5004d19bc5d7b0876382699 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6e473db0b5004d19bc5d7b0876382699. Latest consensus state: current_term: 1 leader_uuid: "6e473db0b5004d19bc5d7b0876382699" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6e473db0b5004d19bc5d7b0876382699" member_type: VOTER last_known_addr { host: "127.19.52.126" port: 43847 } } }
I20260812 06:55:08.287920 23239 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6e473db0b5004d19bc5d7b0876382699 [sys.catalog]: This master's current role is: LEADER
I20260812 06:55:08.287777 23238 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6e473db0b5004d19bc5d7b0876382699 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6e473db0b5004d19bc5d7b0876382699" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6e473db0b5004d19bc5d7b0876382699" member_type: VOTER last_known_addr { host: "127.19.52.126" port: 43847 } } }
I20260812 06:55:08.288968 23238 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6e473db0b5004d19bc5d7b0876382699 [sys.catalog]: This master's current role is: LEADER
W20260812 06:55:08.291373 23252 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 6e473db0b5004d19bc5d7b0876382699: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 06:55:08.291451 23252 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 06:55:08.291533 23246 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 06:55:08.292275 23246 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 06:55:08.298477 23246 catalog_manager.cc:1347] Generated new cluster ID: 421c9844a02546b1ab16d1ca73877d50
I20260812 06:55:08.298575 23246 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 06:55:08.330257 23246 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 06:55:08.331625 23246 catalog_manager.cc:1504] Loading token signing keys...
I20260812 06:55:08.337162 23246 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6e473db0b5004d19bc5d7b0876382699: Generated new TSK 0
I20260812 06:55:08.338030 23246 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.19.52.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.52.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.52.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786517687384214-19665-0/krb5kdc/kudu_127.19.52.65.keytab.
Entry for principal kudu/127.19.52.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786517687384214-19665-0/krb5kdc/kudu_127.19.52.65.keytab.
I20260812 06:55:08.373207 19665 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu
/tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786517687384214-19665-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786517687384214-19665-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786517687384214-19665-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786517687384214-19665-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.52.65:0
--local_ip_for_outbound_sockets=127.19.52.65
--webserver_interface=127.19.52.65
--webserver_port=0
--tserver_master_addrs=127.19.52.126:43847
--keytab_file=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786517687384214-19665-0/krb5kdc/kudu_127.19.52.65.keytab
--principal=kudu/127.19.52.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.52.84:44461
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786517687384214-19665-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786517687384214-19665-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786517687384214-19665-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 06:55:08.492208 23258 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:55:08.492506 23258 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:55:08.492645 23258 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 06:55:08.496798 23258 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 06:55:08.496868 23258 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 06:55:08.496943 23258 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.52.65
I20260812 06:55:08.502432 23258 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.52.84:44461
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786517687384214-19665-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786517687384214-19665-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786517687384214-19665-0/krb5kdc/kudu_127.19.52.65.keytab
--principal=kudu/127.19.52.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.52.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786517687384214-19665-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.19.52.65
--webserver_port=0
--tserver_master_addrs=127.19.52.126:43847
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23258
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.52.65
--log_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786517687384214-19665-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 06:15:54 UTC on ccycloud.quasar-rnfzce.root.comops.site
build id 4581989
I20260812 06:55:08.503532 23258 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:55:08.504662 23258 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 06:55:08.514642 23267 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:55:08.514658 23265 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:55:08.514658 23264 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:55:08.515110 23258 server_base.cc:1034] running on GCE node
I20260812 06:55:08.515534 23258 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:55:08.516230 23258 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:55:08.517393 23258 hybrid_clock.cc:648] HybridClock initialized: now 1786517708517380 us; error 26 us; skew 500 ppm
Aug 12 06:55:08 dist-test-slave-65mx krb5kdc[23136](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517708, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.52.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 06:55:08.520191 23258 init.cc:377] Logged in from keytab as kudu/127.19.52.65@KRBTEST.COM (short username kudu)
I20260812 06:55:08.521155 23258 webserver.cc:460] Webserver started at http://127.19.52.65:45151/ using document root <none> and password file <none>
I20260812 06:55:08.521703 23258 fs_manager.cc:359] Metadata directory not provided
I20260812 06:55:08.521764 23258 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:55:08.521934 23258 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 06:55:08.523993 23258 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786517687384214-19665-0/minicluster-data/ts-0/data/instance:
uuid: "8321719ba635482f9409441e7f38ce60"
format_stamp: "Formatted at 2026-08-12 06:55:08 on dist-test-slave-65mx"
server_key: "cbdfcde6e7c329181036ee521295bfec"
server_key_iv: "736c6132c767f411293787cdad4b276b"
server_key_version: "encryptionkey@0"
I20260812 06:55:08.524609 23258 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786517687384214-19665-0/minicluster-data/ts-0/wal/instance:
uuid: "8321719ba635482f9409441e7f38ce60"
format_stamp: "Formatted at 2026-08-12 06:55:08 on dist-test-slave-65mx"
server_key: "cbdfcde6e7c329181036ee521295bfec"
server_key_iv: "736c6132c767f411293787cdad4b276b"
server_key_version: "encryptionkey@0"
I20260812 06:55:08.528499 23258 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.001s
I20260812 06:55:08.531329 23275 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 06:55:08.532972 23258 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 06:55:08.533183 23258 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786517687384214-19665-0/minicluster-data/ts-0/data,/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786517687384214-19665-0/minicluster-data/ts-0/wal
uuid: "8321719ba635482f9409441e7f38ce60"
format_stamp: "Formatted at 2026-08-12 06:55:08 on dist-test-slave-65mx"
server_key: "cbdfcde6e7c329181036ee521295bfec"
server_key_iv: "736c6132c767f411293787cdad4b276b"
server_key_version: "encryptionkey@0"
I20260812 06:55:08.533373 23258 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786517687384214-19665-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786517687384214-19665-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786517687384214-19665-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:55:08.544137 23258 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 06:55:08.547587 23258 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 06:55:08.547809 23258 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:55:08.548452 23258 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 06:55:08.549407 23258 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 06:55:08.549481 23258 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 06:55:08.549556 23258 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 06:55:08.549597 23258 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 06:55:08.559067 23258 rpc_server.cc:307] RPC server started. Bound to: 127.19.52.65:37253
I20260812 06:55:08.559123 23405 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.52.65:37253 every 8 connection(s)
I20260812 06:55:08.560070 23258 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786517687384214-19665-0/minicluster-data/ts-0/data/info.pb
I20260812 06:55:08.561162 19665 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu as pid 23258
I20260812 06:55:08.561295 19665 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786517687384214-19665-0/minicluster-data/ts-0/wal/instance
I20260812 06:55:08.561590 19665 external_mini_cluster.cc:1428] Setting key e1f5e7cccde903323a1cc47838bf95c6
Aug 12 06:55:08 dist-test-slave-65mx krb5kdc[23136](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517708, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.52.65@KRBTEST.COM for kudu/127.19.52.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.19.52.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.52.66@KRBTEST.COM" created.
I20260812 06:55:08.576433 23235 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:08.563742 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.52.65:34759 (local address 127.19.52.126:43847)
0812 06:55:08.563915 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0812 06:55:08.563922 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 06:55:08.563941 (+ 19us) server_negotiation.cc:407] Connection header received
0812 06:55:08.565002 (+ 1061us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:55:08.565006 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:55:08.565093 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:55:08.565213 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:55:08.566721 (+ 1508us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:08.567274 (+ 553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:08.568515 (+ 1241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:08.568824 (+ 309us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:08.568948 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:55:08.571901 (+ 2953us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:55:08.571904 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:55:08.571906 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:55:08.571939 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:55:08.573801 (+ 1862us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:55:08.574448 (+ 647us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:55:08.574451 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:55:08.574453 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:55:08.574570 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:55:08.575041 (+ 471us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:55:08.575044 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:55:08.575045 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:55:08.575243 (+ 198us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:55:08.575392 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0812 06:55:08.576021 (+ 629us) server_negotiation.cc:299] Negotiation successful
0812 06:55:08.576185 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260812 06:55:08.577291 23408 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:08.562632 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.126:43847 (local address 127.19.52.65:34759)
0812 06:55:08.563224 (+ 592us) negotiation.cc:107] Waiting for socket to connect
0812 06:55:08.563281 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0812 06:55:08.564720 (+ 1439us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:55:08.565526 (+ 806us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:55:08.565536 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:55:08.566053 (+ 517us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:55:08.566568 (+ 515us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:55:08.566581 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:55:08.567416 (+ 835us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:55:08.567420 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:55:08.568363 (+ 943us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:55:08.568373 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:55:08.568999 (+ 626us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:55:08.569003 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:55:08.569188 (+ 185us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:55:08.569850 (+ 662us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 06:55:08.569868 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:55:08.571700 (+ 1832us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 06:55:08.573968 (+ 2268us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:55:08.573974 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:55:08.573989 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:55:08.574323 (+ 334us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:55:08.574720 (+ 397us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:55:08.574726 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:55:08.574729 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:55:08.574928 (+ 199us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:55:08.575431 (+ 503us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 06:55:08.575439 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 06:55:08.575772 (+ 333us) client_negotiation.cc:769] Sending connection context
0812 06:55:08.575948 (+ 176us) client_negotiation.cc:240] Negotiation successful
0812 06:55:08.576248 (+ 300us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":338,"thread_start_us":145,"threads_started":1}
I20260812 06:55:08.578660 23406 heartbeater.cc:347] Connected to a master server at 127.19.52.126:43847
I20260812 06:55:08.578926 23406 heartbeater.cc:464] Registering TS with master...
I20260812 06:55:08.579443 23406 heartbeater.cc:511] Master 127.19.52.126:43847 requested a full tablet report, sending...
I20260812 06:55:08.581218 23192 ts_manager.cc:194] Registered new tserver with Master: 8321719ba635482f9409441e7f38ce60 (127.19.52.65:37253)
I20260812 06:55:08.583194 23192 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.52.65@KRBTEST.COM'} at 127.19.52.65:34759
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.52.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786517687384214-19665-0/krb5kdc/kudu_127.19.52.66.keytab.
Entry for principal kudu/127.19.52.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786517687384214-19665-0/krb5kdc/kudu_127.19.52.66.keytab.
I20260812 06:55:08.594276 19665 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu
/tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786517687384214-19665-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786517687384214-19665-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786517687384214-19665-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786517687384214-19665-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.52.66:0
--local_ip_for_outbound_sockets=127.19.52.66
--webserver_interface=127.19.52.66
--webserver_port=0
--tserver_master_addrs=127.19.52.126:43847
--keytab_file=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786517687384214-19665-0/krb5kdc/kudu_127.19.52.66.keytab
--principal=kudu/127.19.52.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.52.84:44461
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786517687384214-19665-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786517687384214-19665-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786517687384214-19665-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 06:55:08.710247 23413 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:55:08.710611 23413 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:55:08.710682 23413 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 06:55:08.715034 23413 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 06:55:08.715114 23413 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 06:55:08.715176 23413 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.52.66
I20260812 06:55:08.720939 23413 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.52.84:44461
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786517687384214-19665-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786517687384214-19665-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786517687384214-19665-0/krb5kdc/kudu_127.19.52.66.keytab
--principal=kudu/127.19.52.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.52.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786517687384214-19665-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.19.52.66
--webserver_port=0
--tserver_master_addrs=127.19.52.126:43847
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23413
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.52.66
--log_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786517687384214-19665-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 06:15:54 UTC on ccycloud.quasar-rnfzce.root.comops.site
build id 4581989
I20260812 06:55:08.722085 23413 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:55:08.723111 23413 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 06:55:08.732944 23422 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:55:08.732960 23424 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:55:08.733028 23413 server_base.cc:1034] running on GCE node
W20260812 06:55:08.733306 23421 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:55:08.733798 23413 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:55:08.734565 23413 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:55:08.735775 23413 hybrid_clock.cc:648] HybridClock initialized: now 1786517708735740 us; error 49 us; skew 500 ppm
Aug 12 06:55:08 dist-test-slave-65mx krb5kdc[23136](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517708, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.52.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 06:55:08.738961 23413 init.cc:377] Logged in from keytab as kudu/127.19.52.66@KRBTEST.COM (short username kudu)
I20260812 06:55:08.739943 23413 webserver.cc:460] Webserver started at http://127.19.52.66:46561/ using document root <none> and password file <none>
I20260812 06:55:08.740636 23413 fs_manager.cc:359] Metadata directory not provided
I20260812 06:55:08.740739 23413 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:55:08.740978 23413 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 06:55:08.743145 23413 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786517687384214-19665-0/minicluster-data/ts-1/data/instance:
uuid: "c929a274df024d79b09c0da1a5cb9083"
format_stamp: "Formatted at 2026-08-12 06:55:08 on dist-test-slave-65mx"
server_key: "3cdc9ac993b980b20f79265cd2c405c9"
server_key_iv: "b7ed7ea74dab5324a7e11b9722a3a6b1"
server_key_version: "encryptionkey@0"
I20260812 06:55:08.743739 23413 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786517687384214-19665-0/minicluster-data/ts-1/wal/instance:
uuid: "c929a274df024d79b09c0da1a5cb9083"
format_stamp: "Formatted at 2026-08-12 06:55:08 on dist-test-slave-65mx"
server_key: "3cdc9ac993b980b20f79265cd2c405c9"
server_key_iv: "b7ed7ea74dab5324a7e11b9722a3a6b1"
server_key_version: "encryptionkey@0"
I20260812 06:55:08.747738 23413 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260812 06:55:08.750550 23431 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 06:55:08.751883 23413 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260812 06:55:08.752097 23413 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786517687384214-19665-0/minicluster-data/ts-1/data,/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786517687384214-19665-0/minicluster-data/ts-1/wal
uuid: "c929a274df024d79b09c0da1a5cb9083"
format_stamp: "Formatted at 2026-08-12 06:55:08 on dist-test-slave-65mx"
server_key: "3cdc9ac993b980b20f79265cd2c405c9"
server_key_iv: "b7ed7ea74dab5324a7e11b9722a3a6b1"
server_key_version: "encryptionkey@0"
I20260812 06:55:08.752228 23413 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786517687384214-19665-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786517687384214-19665-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786517687384214-19665-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:55:08.785291 23413 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 06:55:08.788983 23413 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 06:55:08.789224 23413 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:55:08.789865 23413 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 06:55:08.790735 23413 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 06:55:08.790803 23413 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 06:55:08.790875 23413 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 06:55:08.790922 23413 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 06:55:08.800545 23413 rpc_server.cc:307] RPC server started. Bound to: 127.19.52.66:33635
I20260812 06:55:08.800599 23566 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.52.66:33635 every 8 connection(s)
I20260812 06:55:08.801514 23413 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786517687384214-19665-0/minicluster-data/ts-1/data/info.pb
I20260812 06:55:08.803406 19665 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu as pid 23413
I20260812 06:55:08.803643 19665 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786517687384214-19665-0/minicluster-data/ts-1/wal/instance
I20260812 06:55:08.803969 19665 external_mini_cluster.cc:1428] Setting key 16f6b0e3b993aa9825530c76f8ee2fe3
Aug 12 06:55:08 dist-test-slave-65mx krb5kdc[23136](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517708, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.52.66@KRBTEST.COM for kudu/127.19.52.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.19.52.67@KRBTEST.COM; defaulting to no policy
I20260812 06:55:08.817967 23235 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:08.803466 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.52.66:33075 (local address 127.19.52.126:43847)
0812 06:55:08.803604 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0812 06:55:08.803608 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 06:55:08.804685 (+ 1077us) server_negotiation.cc:407] Connection header received
0812 06:55:08.806089 (+ 1404us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:55:08.806096 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:55:08.806164 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:55:08.806266 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:55:08.807522 (+ 1256us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:08.808230 (+ 708us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:08.810165 (+ 1935us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:08.810405 (+ 240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:08.810500 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:55:08.813709 (+ 3209us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:55:08.813713 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:55:08.813715 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:55:08.813742 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:55:08.815426 (+ 1684us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:55:08.816044 (+ 618us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:55:08.816047 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:55:08.816049 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:55:08.816142 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:55:08.816664 (+ 522us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:55:08.816668 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:55:08.816669 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:55:08.816878 (+ 209us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:55:08.817009 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0812 06:55:08.817602 (+ 593us) server_negotiation.cc:299] Negotiation successful
0812 06:55:08.817777 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.52.67@KRBTEST.COM" created.
I20260812 06:55:08.818895 23569 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:08.803938 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.126:43847 (local address 127.19.52.66:33075)
0812 06:55:08.804491 (+ 553us) negotiation.cc:107] Waiting for socket to connect
0812 06:55:08.804574 (+ 83us) client_negotiation.cc:174] Beginning negotiation
0812 06:55:08.805864 (+ 1290us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:55:08.806457 (+ 593us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:55:08.806464 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:55:08.806849 (+ 385us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:55:08.807350 (+ 501us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:55:08.807364 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:55:08.808396 (+ 1032us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:55:08.808426 (+ 30us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:55:08.809990 (+ 1564us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:55:08.810006 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:55:08.810603 (+ 597us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:55:08.810613 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:55:08.810891 (+ 278us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:55:08.811596 (+ 705us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 06:55:08.811624 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:55:08.813516 (+ 1892us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 06:55:08.815592 (+ 2076us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:55:08.815598 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:55:08.815616 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:55:08.815914 (+ 298us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:55:08.816239 (+ 325us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:55:08.816242 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:55:08.816244 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:55:08.816500 (+ 256us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:55:08.817010 (+ 510us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 06:55:08.817018 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 06:55:08.817297 (+ 279us) client_negotiation.cc:769] Sending connection context
0812 06:55:08.817510 (+ 213us) client_negotiation.cc:240] Negotiation successful
0812 06:55:08.817878 (+ 368us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":336,"thread_start_us":153,"threads_started":1}
I20260812 06:55:08.820227 23567 heartbeater.cc:347] Connected to a master server at 127.19.52.126:43847
I20260812 06:55:08.820564 23567 heartbeater.cc:464] Registering TS with master...
I20260812 06:55:08.821110 23567 heartbeater.cc:511] Master 127.19.52.126:43847 requested a full tablet report, sending...
I20260812 06:55:08.822283 23192 ts_manager.cc:194] Registered new tserver with Master: c929a274df024d79b09c0da1a5cb9083 (127.19.52.66:33635)
I20260812 06:55:08.822885 23192 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.52.66@KRBTEST.COM'} at 127.19.52.66:33075
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.52.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786517687384214-19665-0/krb5kdc/kudu_127.19.52.67.keytab.
Entry for principal kudu/127.19.52.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786517687384214-19665-0/krb5kdc/kudu_127.19.52.67.keytab.
I20260812 06:55:08.836198 19665 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu
/tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786517687384214-19665-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786517687384214-19665-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786517687384214-19665-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786517687384214-19665-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.52.67:0
--local_ip_for_outbound_sockets=127.19.52.67
--webserver_interface=127.19.52.67
--webserver_port=0
--tserver_master_addrs=127.19.52.126:43847
--keytab_file=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786517687384214-19665-0/krb5kdc/kudu_127.19.52.67.keytab
--principal=kudu/127.19.52.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.52.84:44461
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786517687384214-19665-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786517687384214-19665-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786517687384214-19665-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 06:55:08.951817 23574 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:55:08.952167 23574 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:55:08.952260 23574 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 06:55:08.956496 23574 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 06:55:08.956650 23574 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 06:55:08.956748 23574 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.52.67
I20260812 06:55:08.962378 23574 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.52.84:44461
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786517687384214-19665-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786517687384214-19665-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786517687384214-19665-0/krb5kdc/kudu_127.19.52.67.keytab
--principal=kudu/127.19.52.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.52.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786517687384214-19665-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.19.52.67
--webserver_port=0
--tserver_master_addrs=127.19.52.126:43847
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23574
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.52.67
--log_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786517687384214-19665-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 06:15:54 UTC on ccycloud.quasar-rnfzce.root.comops.site
build id 4581989
I20260812 06:55:08.963559 23574 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:55:08.964654 23574 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 06:55:08.974820 23583 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:55:08.974814 23580 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:55:08.974848 23574 server_base.cc:1034] running on GCE node
W20260812 06:55:08.974869 23581 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:55:08.975508 23574 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:55:08.976161 23574 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:55:08.977341 23574 hybrid_clock.cc:648] HybridClock initialized: now 1786517708977324 us; error 34 us; skew 500 ppm
Aug 12 06:55:08 dist-test-slave-65mx krb5kdc[23136](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517708, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.52.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 06:55:08.980403 23574 init.cc:377] Logged in from keytab as kudu/127.19.52.67@KRBTEST.COM (short username kudu)
I20260812 06:55:08.981436 23574 webserver.cc:460] Webserver started at http://127.19.52.67:35347/ using document root <none> and password file <none>
I20260812 06:55:08.982105 23574 fs_manager.cc:359] Metadata directory not provided
I20260812 06:55:08.982218 23574 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:55:08.982439 23574 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 06:55:08.984822 23574 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786517687384214-19665-0/minicluster-data/ts-2/data/instance:
uuid: "a19e1184b4d54d22b867a484b2e89d45"
format_stamp: "Formatted at 2026-08-12 06:55:08 on dist-test-slave-65mx"
server_key: "859a8927c49dd73451cfb5b1421a6fc3"
server_key_iv: "a3b61dacef9dde4dfdcc24187fcada43"
server_key_version: "encryptionkey@0"
I20260812 06:55:08.985450 23574 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786517687384214-19665-0/minicluster-data/ts-2/wal/instance:
uuid: "a19e1184b4d54d22b867a484b2e89d45"
format_stamp: "Formatted at 2026-08-12 06:55:08 on dist-test-slave-65mx"
server_key: "859a8927c49dd73451cfb5b1421a6fc3"
server_key_iv: "a3b61dacef9dde4dfdcc24187fcada43"
server_key_version: "encryptionkey@0"
I20260812 06:55:08.989599 23574 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260812 06:55:08.992085 23592 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 06:55:08.993399 23574 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260812 06:55:08.993552 23574 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786517687384214-19665-0/minicluster-data/ts-2/data,/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786517687384214-19665-0/minicluster-data/ts-2/wal
uuid: "a19e1184b4d54d22b867a484b2e89d45"
format_stamp: "Formatted at 2026-08-12 06:55:08 on dist-test-slave-65mx"
server_key: "859a8927c49dd73451cfb5b1421a6fc3"
server_key_iv: "a3b61dacef9dde4dfdcc24187fcada43"
server_key_version: "encryptionkey@0"
I20260812 06:55:08.993674 23574 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786517687384214-19665-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786517687384214-19665-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786517687384214-19665-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:55:09.005141 23574 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 06:55:09.008682 23574 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 06:55:09.008898 23574 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:55:09.009531 23574 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 06:55:09.010370 23574 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 06:55:09.010439 23574 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 06:55:09.010504 23574 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 06:55:09.010545 23574 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 06:55:09.019480 23574 rpc_server.cc:307] RPC server started. Bound to: 127.19.52.67:36853
I20260812 06:55:09.019531 23723 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.52.67:36853 every 8 connection(s)
I20260812 06:55:09.020443 23574 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786517687384214-19665-0/minicluster-data/ts-2/data/info.pb
I20260812 06:55:09.024432 19665 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu as pid 23574
I20260812 06:55:09.024598 19665 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786517687384214-19665-0/minicluster-data/ts-2/wal/instance
I20260812 06:55:09.024937 19665 external_mini_cluster.cc:1428] Setting key afb0a30deeb7fd1e7be59f9b683045e9
Aug 12 06:55:09 dist-test-slave-65mx krb5kdc[23136](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517708, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.52.67@KRBTEST.COM for kudu/127.19.52.126@KRBTEST.COM
I20260812 06:55:09.037118 23235 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:09.022456 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.52.67:54821 (local address 127.19.52.126:43847)
0812 06:55:09.022597 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0812 06:55:09.022600 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 06:55:09.023479 (+ 879us) server_negotiation.cc:407] Connection header received
0812 06:55:09.025591 (+ 2112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:55:09.025595 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:55:09.025654 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:55:09.025738 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:55:09.027411 (+ 1673us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:09.028002 (+ 591us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:09.029523 (+ 1521us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:09.029747 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:09.029841 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:55:09.032719 (+ 2878us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:55:09.032722 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:55:09.032724 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:55:09.032753 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:55:09.034536 (+ 1783us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:55:09.035131 (+ 595us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:55:09.035134 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:55:09.035136 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:55:09.035226 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:55:09.035606 (+ 380us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:55:09.035609 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:55:09.035610 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:55:09.035815 (+ 205us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:55:09.035904 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0812 06:55:09.036757 (+ 853us) server_negotiation.cc:299] Negotiation successful
0812 06:55:09.036875 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260812 06:55:09.038064 23727 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:09.022817 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.126:43847 (local address 127.19.52.67:54821)
0812 06:55:09.023288 (+ 471us) negotiation.cc:107] Waiting for socket to connect
0812 06:55:09.023332 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0812 06:55:09.025170 (+ 1838us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:55:09.026054 (+ 884us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:55:09.026101 (+ 47us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:55:09.026538 (+ 437us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:55:09.027273 (+ 735us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:55:09.027295 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:55:09.028144 (+ 849us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:55:09.028149 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:55:09.029355 (+ 1206us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:55:09.029367 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:55:09.029927 (+ 560us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:55:09.029930 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:55:09.030111 (+ 181us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:55:09.030849 (+ 738us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 06:55:09.030868 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:55:09.032481 (+ 1613us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 06:55:09.034659 (+ 2178us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:55:09.034674 (+ 15us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:55:09.034689 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:55:09.034990 (+ 301us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:55:09.035315 (+ 325us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:55:09.035317 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:55:09.035318 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:55:09.035481 (+ 163us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:55:09.035955 (+ 474us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 06:55:09.035961 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 06:55:09.036285 (+ 324us) client_negotiation.cc:769] Sending connection context
0812 06:55:09.036617 (+ 332us) client_negotiation.cc:240] Negotiation successful
0812 06:55:09.037065 (+ 448us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":287,"thread_start_us":119,"threads_started":1}
I20260812 06:55:09.039645 23724 heartbeater.cc:347] Connected to a master server at 127.19.52.126:43847
I20260812 06:55:09.039921 23724 heartbeater.cc:464] Registering TS with master...
I20260812 06:55:09.040606 23724 heartbeater.cc:511] Master 127.19.52.126:43847 requested a full tablet report, sending...
I20260812 06:55:09.041716 23191 ts_manager.cc:194] Registered new tserver with Master: a19e1184b4d54d22b867a484b2e89d45 (127.19.52.67:36853)
I20260812 06:55:09.042367 23191 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.52.67@KRBTEST.COM'} at 127.19.52.67:54821
I20260812 06:55:09.052536 19665 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 12 06:55:09 dist-test-slave-65mx krb5kdc[23136](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517709, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for joe-interloper@KRBTEST.COM:
Aug 12 06:55:09 dist-test-slave-65mx krb5kdc[23136](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517709, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.19.52.126@KRBTEST.COM
I20260812 06:55:09.082401 23235 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:09.072941 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55102 (local address 127.19.52.126:43847)
0812 06:55:09.073126 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0812 06:55:09.073132 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 06:55:09.073297 (+ 165us) server_negotiation.cc:407] Connection header received
0812 06:55:09.073472 (+ 175us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:55:09.073475 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:55:09.073529 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:55:09.073641 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:55:09.074419 (+ 778us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:09.074955 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:09.075805 (+ 850us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:09.076032 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:09.076146 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:55:09.079154 (+ 3008us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:55:09.079157 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:55:09.079159 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:55:09.079192 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:55:09.080607 (+ 1415us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:55:09.081053 (+ 446us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:55:09.081057 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:55:09.081059 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:55:09.081146 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:55:09.081430 (+ 284us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:55:09.081434 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:55:09.081436 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:55:09.081686 (+ 250us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:55:09.081836 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0812 06:55:09.082100 (+ 264us) server_negotiation.cc:299] Negotiation successful
0812 06:55:09.082233 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
W20260812 06:55:09.082914 23191 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.ConnectToMaster from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:55102
I20260812 06:55:09.084252 19665 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu with pid 23258
I20260812 06:55:09.093643 19665 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu with pid 23413
I20260812 06:55:09.104843 19665 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu with pid 23574
I20260812 06:55:09.114339 19665 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu with pid 23152
2026-08-12T06:55:09Z chronyd exiting
[ OK ] SecurityITest.TestUnauthorizedClientKerberosCredentials (2345 ms)
[ RUN ] SecurityITest.TestAuthorizedSuperuser
Loading random data
Initializing database '/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786517687384214-19665-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 06:55:09 dist-test-slave-65mx krb5kdc[23738](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 06:55:09 dist-test-slave-65mx krb5kdc[23738](info): set up 2 sockets
Aug 12 06:55:09 dist-test-slave-65mx krb5kdc[23738](info): commencing operation
krb5kdc: starting...
W20260812 06:55:10.279856 19665 mini_kdc.cc:121] Time spent starting KDC: real 1.139s user 0.006s sys 0.001s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786517687384214-19665-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786517687384214-19665-0/krb5kdc/test-user.keytab.
Aug 12 06:55:10 dist-test-slave-65mx krb5kdc[23738](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517710, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T06:55:10Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T06:55:10Z Disabled control of system clock
WARNING: no policy specified for kudu/127.19.52.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.52.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.52.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786517687384214-19665-0/krb5kdc/kudu_127.19.52.126.keytab.
Entry for principal kudu/127.19.52.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786517687384214-19665-0/krb5kdc/kudu_127.19.52.126.keytab.
I20260812 06:55:10.421641 19665 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu
/tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786517687384214-19665-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786517687384214-19665-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786517687384214-19665-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786517687384214-19665-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.52.126:34489
--webserver_interface=127.19.52.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786517687384214-19665-0/krb5kdc/kudu_127.19.52.126.keytab
--principal=kudu/127.19.52.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.52.84:36481
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.19.52.126:34489
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786517687384214-19665-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786517687384214-19665-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786517687384214-19665-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 06:55:10.561956 23756 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:55:10.562273 23756 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:55:10.562318 23756 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 06:55:10.566952 23756 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 06:55:10.567044 23756 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 06:55:10.567065 23756 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 06:55:10.567082 23756 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 06:55:10.567098 23756 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 06:55:10.573050 23756 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.52.84:36481
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786517687384214-19665-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786517687384214-19665-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.19.52.126:34489
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786517687384214-19665-0/krb5kdc/kudu_127.19.52.126.keytab
--principal=kudu/127.19.52.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.52.126:34489
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786517687384214-19665-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.19.52.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23756
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786517687384214-19665-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 06:15:54 UTC on ccycloud.quasar-rnfzce.root.comops.site
build id 4581989
I20260812 06:55:10.574313 23756 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:55:10.575433 23756 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 06:55:10.585325 23765 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:55:10.585320 23761 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:55:10.585320 23762 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:55:10.585829 23756 server_base.cc:1034] running on GCE node
I20260812 06:55:10.586474 23756 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:55:10.587859 23756 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:55:10.589066 23756 hybrid_clock.cc:648] HybridClock initialized: now 1786517710589026 us; error 46 us; skew 500 ppm
Aug 12 06:55:10 dist-test-slave-65mx krb5kdc[23738](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517710, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.52.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 06:55:10.592137 23756 init.cc:377] Logged in from keytab as kudu/127.19.52.126@KRBTEST.COM (short username kudu)
I20260812 06:55:10.593175 23756 webserver.cc:460] Webserver started at http://127.19.52.126:38215/ using document root <none> and password file <none>
I20260812 06:55:10.593737 23756 fs_manager.cc:359] Metadata directory not provided
I20260812 06:55:10.593802 23756 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:55:10.593987 23756 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 06:55:10.596247 23756 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786517687384214-19665-0/minicluster-data/master-0/data/instance:
uuid: "ffddc8688a46403bbb4b8610a790581a"
format_stamp: "Formatted at 2026-08-12 06:55:10 on dist-test-slave-65mx"
server_key: "c8234320232d72a48f44e6581725a98b"
server_key_iv: "6ba8efc0fb684e979e59878eb93c6f6b"
server_key_version: "encryptionkey@0"
I20260812 06:55:10.596935 23756 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786517687384214-19665-0/minicluster-data/master-0/wal/instance:
uuid: "ffddc8688a46403bbb4b8610a790581a"
format_stamp: "Formatted at 2026-08-12 06:55:10 on dist-test-slave-65mx"
server_key: "c8234320232d72a48f44e6581725a98b"
server_key_iv: "6ba8efc0fb684e979e59878eb93c6f6b"
server_key_version: "encryptionkey@0"
I20260812 06:55:10.601639 23756 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260812 06:55:10.604691 23773 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 06:55:10.606163 23756 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 06:55:10.606293 23756 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786517687384214-19665-0/minicluster-data/master-0/data,/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786517687384214-19665-0/minicluster-data/master-0/wal
uuid: "ffddc8688a46403bbb4b8610a790581a"
format_stamp: "Formatted at 2026-08-12 06:55:10 on dist-test-slave-65mx"
server_key: "c8234320232d72a48f44e6581725a98b"
server_key_iv: "6ba8efc0fb684e979e59878eb93c6f6b"
server_key_version: "encryptionkey@0"
I20260812 06:55:10.606403 23756 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786517687384214-19665-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786517687384214-19665-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786517687384214-19665-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:55:10.628494 23756 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 06:55:10.632066 23756 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 06:55:10.632251 23756 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:55:10.640600 23756 rpc_server.cc:307] RPC server started. Bound to: 127.19.52.126:34489
I20260812 06:55:10.640661 23833 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.52.126:34489 every 8 connection(s)
I20260812 06:55:10.641712 23756 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786517687384214-19665-0/minicluster-data/master-0/data/info.pb
I20260812 06:55:10.645241 23834 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 06:55:10.650061 19665 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu as pid 23756
I20260812 06:55:10.650202 19665 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786517687384214-19665-0/minicluster-data/master-0/wal/instance
I20260812 06:55:10.650484 19665 external_mini_cluster.cc:1428] Setting key e209690a0907588ea56ecc723d0f83a1
I20260812 06:55:10.652117 23834 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ffddc8688a46403bbb4b8610a790581a: Bootstrap starting.
I20260812 06:55:10.655131 23834 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ffddc8688a46403bbb4b8610a790581a: Neither blocks nor log segments found. Creating new log.
I20260812 06:55:10.656498 23834 log.cc:824] T 00000000000000000000000000000000 P ffddc8688a46403bbb4b8610a790581a: Log is configured to *not* fsync() on all Append() calls
I20260812 06:55:10.659358 23834 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ffddc8688a46403bbb4b8610a790581a: No bootstrap required, opened a new log
Aug 12 06:55:10 dist-test-slave-65mx krb5kdc[23738](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517710, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.19.52.126@KRBTEST.COM
I20260812 06:55:10.664371 23834 raft_consensus.cc:348] T 00000000000000000000000000000000 P ffddc8688a46403bbb4b8610a790581a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ffddc8688a46403bbb4b8610a790581a" member_type: VOTER last_known_addr { host: "127.19.52.126" port: 34489 } }
I20260812 06:55:10.664762 23834 raft_consensus.cc:374] T 00000000000000000000000000000000 P ffddc8688a46403bbb4b8610a790581a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 06:55:10.664863 23834 raft_consensus.cc:729] T 00000000000000000000000000000000 P ffddc8688a46403bbb4b8610a790581a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ffddc8688a46403bbb4b8610a790581a, State: Initialized, Role: FOLLOWER
I20260812 06:55:10.665416 23834 consensus_queue.cc:260] T 00000000000000000000000000000000 P ffddc8688a46403bbb4b8610a790581a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ffddc8688a46403bbb4b8610a790581a" member_type: VOTER last_known_addr { host: "127.19.52.126" port: 34489 } }
I20260812 06:55:10.665602 23834 raft_consensus.cc:388] T 00000000000000000000000000000000 P ffddc8688a46403bbb4b8610a790581a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 06:55:10.665697 23834 raft_consensus.cc:482] T 00000000000000000000000000000000 P ffddc8688a46403bbb4b8610a790581a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 06:55:10.665808 23834 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ffddc8688a46403bbb4b8610a790581a [term 0 FOLLOWER]: Advancing to term 1
I20260812 06:55:10.668260 23837 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:10.652035 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46188 (local address 127.19.52.126:34489)
0812 06:55:10.652533 (+ 498us) server_negotiation.cc:206] Beginning negotiation
0812 06:55:10.652574 (+ 41us) server_negotiation.cc:399] Waiting for connection header
0812 06:55:10.652620 (+ 46us) server_negotiation.cc:407] Connection header received
0812 06:55:10.653518 (+ 898us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:55:10.653545 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:55:10.654048 (+ 503us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:55:10.654443 (+ 395us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:55:10.655272 (+ 829us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:10.657204 (+ 1932us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:10.658307 (+ 1103us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:10.659050 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:10.659231 (+ 181us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:55:10.662045 (+ 2814us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:55:10.662054 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:55:10.662076 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:55:10.662128 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:55:10.664425 (+ 2297us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:55:10.665239 (+ 814us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:55:10.665245 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:55:10.665251 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:55:10.665410 (+ 159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:55:10.665805 (+ 395us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:55:10.665809 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:55:10.665811 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:55:10.666392 (+ 581us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:55:10.666621 (+ 229us) server_negotiation.cc:1035] Waiting for connection context
0812 06:55:10.666873 (+ 252us) server_negotiation.cc:299] Negotiation successful
0812 06:55:10.667226 (+ 353us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":333,"thread_start_us":170,"threads_started":1}
I20260812 06:55:10.739123 23834 raft_consensus.cc:504] T 00000000000000000000000000000000 P ffddc8688a46403bbb4b8610a790581a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ffddc8688a46403bbb4b8610a790581a" member_type: VOTER last_known_addr { host: "127.19.52.126" port: 34489 } }
I20260812 06:55:10.739699 23834 leader_election.cc:302] T 00000000000000000000000000000000 P ffddc8688a46403bbb4b8610a790581a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ffddc8688a46403bbb4b8610a790581a; no voters:
I20260812 06:55:10.740152 23834 leader_election.cc:288] T 00000000000000000000000000000000 P ffddc8688a46403bbb4b8610a790581a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 06:55:10.740330 23839 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ffddc8688a46403bbb4b8610a790581a [term 1 FOLLOWER]: Leader election won for term 1
I20260812 06:55:10.740644 23839 raft_consensus.cc:686] T 00000000000000000000000000000000 P ffddc8688a46403bbb4b8610a790581a [term 1 LEADER]: Becoming Leader. State: Replica: ffddc8688a46403bbb4b8610a790581a, State: Running, Role: LEADER
I20260812 06:55:10.741026 23839 consensus_queue.cc:237] T 00000000000000000000000000000000 P ffddc8688a46403bbb4b8610a790581a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ffddc8688a46403bbb4b8610a790581a" member_type: VOTER last_known_addr { host: "127.19.52.126" port: 34489 } }
I20260812 06:55:10.741819 23834 sys_catalog.cc:564] T 00000000000000000000000000000000 P ffddc8688a46403bbb4b8610a790581a [sys.catalog]: configured and running, proceeding with master startup.
I20260812 06:55:10.742496 23841 sys_catalog.cc:455] T 00000000000000000000000000000000 P ffddc8688a46403bbb4b8610a790581a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ffddc8688a46403bbb4b8610a790581a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ffddc8688a46403bbb4b8610a790581a" member_type: VOTER last_known_addr { host: "127.19.52.126" port: 34489 } } }
I20260812 06:55:10.742647 23841 sys_catalog.cc:458] T 00000000000000000000000000000000 P ffddc8688a46403bbb4b8610a790581a [sys.catalog]: This master's current role is: LEADER
I20260812 06:55:10.743003 23840 sys_catalog.cc:455] T 00000000000000000000000000000000 P ffddc8688a46403bbb4b8610a790581a [sys.catalog]: SysCatalogTable state changed. Reason: New leader ffddc8688a46403bbb4b8610a790581a. Latest consensus state: current_term: 1 leader_uuid: "ffddc8688a46403bbb4b8610a790581a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ffddc8688a46403bbb4b8610a790581a" member_type: VOTER last_known_addr { host: "127.19.52.126" port: 34489 } } }
I20260812 06:55:10.743187 23840 sys_catalog.cc:458] T 00000000000000000000000000000000 P ffddc8688a46403bbb4b8610a790581a [sys.catalog]: This master's current role is: LEADER
I20260812 06:55:10.743677 23848 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 06:55:10.747452 23848 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 06:55:10.755292 23848 catalog_manager.cc:1347] Generated new cluster ID: de3a8ce560cc41038cfdd3b4dd81e287
I20260812 06:55:10.755393 23848 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 06:55:10.767141 23848 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 06:55:10.768503 23848 catalog_manager.cc:1504] Loading token signing keys...
I20260812 06:55:10.781629 23848 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ffddc8688a46403bbb4b8610a790581a: Generated new TSK 0
I20260812 06:55:10.782629 23848 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.19.52.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.52.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.52.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786517687384214-19665-0/krb5kdc/kudu_127.19.52.65.keytab.
Entry for principal kudu/127.19.52.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786517687384214-19665-0/krb5kdc/kudu_127.19.52.65.keytab.
W20260812 06:55:11.685415 19665 mini_kdc.cc:262] Time spent creating service keytab for kudu/127.19.52.65: real 0.886s user 0.000s sys 0.005s
I20260812 06:55:11.686023 19665 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu
/tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786517687384214-19665-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786517687384214-19665-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786517687384214-19665-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786517687384214-19665-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.52.65:0
--local_ip_for_outbound_sockets=127.19.52.65
--webserver_interface=127.19.52.65
--webserver_port=0
--tserver_master_addrs=127.19.52.126:34489
--keytab_file=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786517687384214-19665-0/krb5kdc/kudu_127.19.52.65.keytab
--principal=kudu/127.19.52.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.52.84:36481
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786517687384214-19665-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786517687384214-19665-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786517687384214-19665-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 06:55:11.806787 23869 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:55:11.807171 23869 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:55:11.807224 23869 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 06:55:11.812179 23869 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 06:55:11.812269 23869 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 06:55:11.812377 23869 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.52.65
I20260812 06:55:11.818554 23869 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.52.84:36481
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786517687384214-19665-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786517687384214-19665-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786517687384214-19665-0/krb5kdc/kudu_127.19.52.65.keytab
--principal=kudu/127.19.52.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.52.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786517687384214-19665-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.19.52.65
--webserver_port=0
--tserver_master_addrs=127.19.52.126:34489
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23869
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.52.65
--log_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786517687384214-19665-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 06:15:54 UTC on ccycloud.quasar-rnfzce.root.comops.site
build id 4581989
I20260812 06:55:11.819850 23869 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:55:11.821036 23869 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 06:55:11.831406 23874 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:55:11.831406 23878 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:55:11.831450 23875 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:55:11.831708 23869 server_base.cc:1034] running on GCE node
I20260812 06:55:11.832372 23869 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:55:11.833184 23869 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:55:11.834401 23869 hybrid_clock.cc:648] HybridClock initialized: now 1786517711834355 us; error 53 us; skew 500 ppm
Aug 12 06:55:11 dist-test-slave-65mx krb5kdc[23738](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517711, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.52.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 06:55:11.837659 23869 init.cc:377] Logged in from keytab as kudu/127.19.52.65@KRBTEST.COM (short username kudu)
I20260812 06:55:11.838645 23869 webserver.cc:460] Webserver started at http://127.19.52.65:37333/ using document root <none> and password file <none>
I20260812 06:55:11.839246 23869 fs_manager.cc:359] Metadata directory not provided
I20260812 06:55:11.839340 23869 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:55:11.839586 23869 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 06:55:11.841925 23869 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786517687384214-19665-0/minicluster-data/ts-0/data/instance:
uuid: "87896cf24cf54a5c8052d0ed5d6cddcd"
format_stamp: "Formatted at 2026-08-12 06:55:11 on dist-test-slave-65mx"
server_key: "283163618219c0de44749b4555086c6f"
server_key_iv: "b197e3f12f6503b03eeecc04d7eff9cc"
server_key_version: "encryptionkey@0"
I20260812 06:55:11.842602 23869 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786517687384214-19665-0/minicluster-data/ts-0/wal/instance:
uuid: "87896cf24cf54a5c8052d0ed5d6cddcd"
format_stamp: "Formatted at 2026-08-12 06:55:11 on dist-test-slave-65mx"
server_key: "283163618219c0de44749b4555086c6f"
server_key_iv: "b197e3f12f6503b03eeecc04d7eff9cc"
server_key_version: "encryptionkey@0"
I20260812 06:55:11.847311 23869 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.004s
I20260812 06:55:11.850236 23887 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 06:55:11.851555 23869 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260812 06:55:11.851707 23869 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786517687384214-19665-0/minicluster-data/ts-0/data,/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786517687384214-19665-0/minicluster-data/ts-0/wal
uuid: "87896cf24cf54a5c8052d0ed5d6cddcd"
format_stamp: "Formatted at 2026-08-12 06:55:11 on dist-test-slave-65mx"
server_key: "283163618219c0de44749b4555086c6f"
server_key_iv: "b197e3f12f6503b03eeecc04d7eff9cc"
server_key_version: "encryptionkey@0"
I20260812 06:55:11.851835 23869 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786517687384214-19665-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786517687384214-19665-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786517687384214-19665-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:55:11.865944 23869 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 06:55:11.869946 23869 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 06:55:11.870196 23869 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:55:11.870872 23869 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 06:55:11.872196 23869 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 06:55:11.872269 23869 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 06:55:11.872334 23869 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 06:55:11.872378 23869 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 06:55:11.883839 23869 rpc_server.cc:307] RPC server started. Bound to: 127.19.52.65:43519
I20260812 06:55:11.883922 24025 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.52.65:43519 every 8 connection(s)
I20260812 06:55:11.884982 23869 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786517687384214-19665-0/minicluster-data/ts-0/data/info.pb
I20260812 06:55:11.894599 19665 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu as pid 23869
I20260812 06:55:11.894806 19665 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786517687384214-19665-0/minicluster-data/ts-0/wal/instance
I20260812 06:55:11.895161 19665 external_mini_cluster.cc:1428] Setting key 021b494ba833eaf46e5eb16f7f224645
Aug 12 06:55:11 dist-test-slave-65mx krb5kdc[23738](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517711, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.52.65@KRBTEST.COM for kudu/127.19.52.126@KRBTEST.COM
I20260812 06:55:11.903259 24028 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:11.887589 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.52.65:60085 (local address 127.19.52.126:34489)
0812 06:55:11.887903 (+ 314us) server_negotiation.cc:206] Beginning negotiation
0812 06:55:11.887907 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 06:55:11.888651 (+ 744us) server_negotiation.cc:407] Connection header received
0812 06:55:11.891031 (+ 2380us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:55:11.891034 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:55:11.891107 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:55:11.891205 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:55:11.892833 (+ 1628us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:11.893469 (+ 636us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:11.894722 (+ 1253us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:11.895077 (+ 355us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:11.895195 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:55:11.898308 (+ 3113us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:55:11.898312 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:55:11.898315 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:55:11.898350 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:55:11.900324 (+ 1974us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:55:11.901043 (+ 719us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:55:11.901049 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:55:11.901054 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:55:11.901163 (+ 109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:55:11.901620 (+ 457us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:55:11.901626 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:55:11.901631 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:55:11.901930 (+ 299us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:55:11.902059 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0812 06:55:11.902801 (+ 742us) server_negotiation.cc:299] Negotiation successful
0812 06:55:11.902964 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":228,"thread_start_us":129,"threads_started":1}
I20260812 06:55:11.904048 24029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:11.887877 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.126:34489 (local address 127.19.52.65:60085)
0812 06:55:11.888415 (+ 538us) negotiation.cc:107] Waiting for socket to connect
0812 06:55:11.888485 (+ 70us) client_negotiation.cc:174] Beginning negotiation
0812 06:55:11.890752 (+ 2267us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:55:11.891401 (+ 649us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:55:11.891409 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:55:11.891822 (+ 413us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:55:11.892507 (+ 685us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:55:11.892524 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:55:11.893617 (+ 1093us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:55:11.893621 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:55:11.894573 (+ 952us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:55:11.894584 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:55:11.895264 (+ 680us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:55:11.895269 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:55:11.895486 (+ 217us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:55:11.896178 (+ 692us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 06:55:11.896196 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:55:11.898122 (+ 1926us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 06:55:11.900471 (+ 2349us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:55:11.900476 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:55:11.900514 (+ 38us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:55:11.900892 (+ 378us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:55:11.901288 (+ 396us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:55:11.901291 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:55:11.901292 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:55:11.901493 (+ 201us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:55:11.902073 (+ 580us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 06:55:11.902080 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 06:55:11.902459 (+ 379us) client_negotiation.cc:769] Sending connection context
0812 06:55:11.902682 (+ 223us) client_negotiation.cc:240] Negotiation successful
0812 06:55:11.903008 (+ 326us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":312,"thread_start_us":142,"threads_started":1}
I20260812 06:55:11.905622 24026 heartbeater.cc:347] Connected to a master server at 127.19.52.126:34489
I20260812 06:55:11.905954 24026 heartbeater.cc:464] Registering TS with master...
I20260812 06:55:11.906675 24026 heartbeater.cc:511] Master 127.19.52.126:34489 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.19.52.66@KRBTEST.COM; defaulting to no policy
I20260812 06:55:11.908648 23790 ts_manager.cc:194] Registered new tserver with Master: 87896cf24cf54a5c8052d0ed5d6cddcd (127.19.52.65:43519)
I20260812 06:55:11.909870 23790 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.52.65@KRBTEST.COM'} at 127.19.52.65:60085
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.52.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.52.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786517687384214-19665-0/krb5kdc/kudu_127.19.52.66.keytab.
Entry for principal kudu/127.19.52.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786517687384214-19665-0/krb5kdc/kudu_127.19.52.66.keytab.
W20260812 06:55:12.341707 19665 mini_kdc.cc:262] Time spent creating service keytab for kudu/127.19.52.66: real 0.445s user 0.000s sys 0.005s
I20260812 06:55:12.342392 19665 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu
/tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786517687384214-19665-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786517687384214-19665-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786517687384214-19665-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786517687384214-19665-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.52.66:0
--local_ip_for_outbound_sockets=127.19.52.66
--webserver_interface=127.19.52.66
--webserver_port=0
--tserver_master_addrs=127.19.52.126:34489
--keytab_file=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786517687384214-19665-0/krb5kdc/kudu_127.19.52.66.keytab
--principal=kudu/127.19.52.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.52.84:36481
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786517687384214-19665-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786517687384214-19665-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786517687384214-19665-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 06:55:12.464365 24034 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:55:12.464823 24034 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:55:12.464880 24034 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 06:55:12.469396 24034 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 06:55:12.469480 24034 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 06:55:12.469545 24034 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.52.66
I20260812 06:55:12.475330 24034 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.52.84:36481
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786517687384214-19665-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786517687384214-19665-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786517687384214-19665-0/krb5kdc/kudu_127.19.52.66.keytab
--principal=kudu/127.19.52.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.52.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786517687384214-19665-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.19.52.66
--webserver_port=0
--tserver_master_addrs=127.19.52.126:34489
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24034
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.52.66
--log_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786517687384214-19665-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 06:15:54 UTC on ccycloud.quasar-rnfzce.root.comops.site
build id 4581989
I20260812 06:55:12.476624 24034 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:55:12.477746 24034 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 06:55:12.488418 24039 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:55:12.488632 24040 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:55:12.488348 24042 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:55:12.489013 24034 server_base.cc:1034] running on GCE node
I20260812 06:55:12.489503 24034 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:55:12.490301 24034 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:55:12.491544 24034 hybrid_clock.cc:648] HybridClock initialized: now 1786517712491507 us; error 49 us; skew 500 ppm
Aug 12 06:55:12 dist-test-slave-65mx krb5kdc[23738](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517712, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.52.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 06:55:12.494796 24034 init.cc:377] Logged in from keytab as kudu/127.19.52.66@KRBTEST.COM (short username kudu)
I20260812 06:55:12.495813 24034 webserver.cc:460] Webserver started at http://127.19.52.66:38513/ using document root <none> and password file <none>
I20260812 06:55:12.496419 24034 fs_manager.cc:359] Metadata directory not provided
I20260812 06:55:12.496529 24034 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:55:12.496789 24034 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 06:55:12.499063 24034 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786517687384214-19665-0/minicluster-data/ts-1/data/instance:
uuid: "f76ddf82fb0346469afe8f435d9891c9"
format_stamp: "Formatted at 2026-08-12 06:55:12 on dist-test-slave-65mx"
server_key: "167cf840b0736be702aa1e0d5050d4ef"
server_key_iv: "8a9ee5f3585a76581bcdc1633286020b"
server_key_version: "encryptionkey@0"
I20260812 06:55:12.499692 24034 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786517687384214-19665-0/minicluster-data/ts-1/wal/instance:
uuid: "f76ddf82fb0346469afe8f435d9891c9"
format_stamp: "Formatted at 2026-08-12 06:55:12 on dist-test-slave-65mx"
server_key: "167cf840b0736be702aa1e0d5050d4ef"
server_key_iv: "8a9ee5f3585a76581bcdc1633286020b"
server_key_version: "encryptionkey@0"
I20260812 06:55:12.504065 24034 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.003s
I20260812 06:55:12.507022 24054 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 06:55:12.508510 24034 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260812 06:55:12.508726 24034 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786517687384214-19665-0/minicluster-data/ts-1/data,/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786517687384214-19665-0/minicluster-data/ts-1/wal
uuid: "f76ddf82fb0346469afe8f435d9891c9"
format_stamp: "Formatted at 2026-08-12 06:55:12 on dist-test-slave-65mx"
server_key: "167cf840b0736be702aa1e0d5050d4ef"
server_key_iv: "8a9ee5f3585a76581bcdc1633286020b"
server_key_version: "encryptionkey@0"
I20260812 06:55:12.508860 24034 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786517687384214-19665-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786517687384214-19665-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786517687384214-19665-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:55:12.527877 24034 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 06:55:12.532016 24034 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 06:55:12.532248 24034 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:55:12.532950 24034 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 06:55:12.533828 24034 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 06:55:12.533896 24034 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 06:55:12.533969 24034 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 06:55:12.534019 24034 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 06:55:12.543500 24034 rpc_server.cc:307] RPC server started. Bound to: 127.19.52.66:43053
I20260812 06:55:12.543514 24189 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.52.66:43053 every 8 connection(s)
I20260812 06:55:12.544647 24034 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786517687384214-19665-0/minicluster-data/ts-1/data/info.pb
I20260812 06:55:12.551146 19665 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu as pid 24034
I20260812 06:55:12.551313 19665 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786517687384214-19665-0/minicluster-data/ts-1/wal/instance
I20260812 06:55:12.551659 19665 external_mini_cluster.cc:1428] Setting key 3c56d26a9a5941cd288034277a7afec5
Aug 12 06:55:12 dist-test-slave-65mx krb5kdc[23738](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517712, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.52.66@KRBTEST.COM for kudu/127.19.52.126@KRBTEST.COM
I20260812 06:55:12.561239 24192 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:12.546575 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.52.66:59333 (local address 127.19.52.126:34489)
0812 06:55:12.546834 (+ 259us) server_negotiation.cc:206] Beginning negotiation
0812 06:55:12.546838 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 06:55:12.547574 (+ 736us) server_negotiation.cc:407] Connection header received
0812 06:55:12.549122 (+ 1548us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:55:12.549126 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:55:12.549192 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:55:12.549315 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:55:12.550566 (+ 1251us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:12.551131 (+ 565us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:12.552871 (+ 1740us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:12.553183 (+ 312us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:12.553314 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:55:12.556304 (+ 2990us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:55:12.556307 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:55:12.556309 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:55:12.556379 (+ 70us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:55:12.558179 (+ 1800us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:55:12.559006 (+ 827us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:55:12.559009 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:55:12.559011 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:55:12.559167 (+ 156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:55:12.559683 (+ 516us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:55:12.559686 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:55:12.559688 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:55:12.559917 (+ 229us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:55:12.560072 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0812 06:55:12.560915 (+ 843us) server_negotiation.cc:299] Negotiation successful
0812 06:55:12.561083 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":177,"thread_start_us":94,"threads_started":1}
I20260812 06:55:12.562721 24193 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:12.546966 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.126:34489 (local address 127.19.52.66:59333)
0812 06:55:12.547420 (+ 454us) negotiation.cc:107] Waiting for socket to connect
0812 06:55:12.547464 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0812 06:55:12.548890 (+ 1426us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:55:12.549501 (+ 611us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:55:12.549507 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:55:12.549878 (+ 371us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:55:12.550381 (+ 503us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:55:12.550394 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:55:12.551273 (+ 879us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:55:12.551279 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:55:12.552736 (+ 1457us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:55:12.552747 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:55:12.553366 (+ 619us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:55:12.553370 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:55:12.553577 (+ 207us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:55:12.554266 (+ 689us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 06:55:12.554284 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:55:12.556068 (+ 1784us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 06:55:12.558347 (+ 2279us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:55:12.558358 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:55:12.558381 (+ 23us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:55:12.558840 (+ 459us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:55:12.559297 (+ 457us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:55:12.559301 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:55:12.559304 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:55:12.559542 (+ 238us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:55:12.560103 (+ 561us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 06:55:12.560113 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 06:55:12.560536 (+ 423us) client_negotiation.cc:769] Sending connection context
0812 06:55:12.560827 (+ 291us) client_negotiation.cc:240] Negotiation successful
0812 06:55:12.561263 (+ 436us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":270,"thread_start_us":123,"threads_started":1}
WARNING: no policy specified for kudu/127.19.52.67@KRBTEST.COM; defaulting to no policy
I20260812 06:55:12.564216 24190 heartbeater.cc:347] Connected to a master server at 127.19.52.126:34489
I20260812 06:55:12.564517 24190 heartbeater.cc:464] Registering TS with master...
I20260812 06:55:12.565178 24190 heartbeater.cc:511] Master 127.19.52.126:34489 requested a full tablet report, sending...
I20260812 06:55:12.566536 23790 ts_manager.cc:194] Registered new tserver with Master: f76ddf82fb0346469afe8f435d9891c9 (127.19.52.66:43053)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.52.67@KRBTEST.COM" created.
I20260812 06:55:12.567286 23790 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.52.66@KRBTEST.COM'} at 127.19.52.66:59333
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.52.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786517687384214-19665-0/krb5kdc/kudu_127.19.52.67.keytab.
Entry for principal kudu/127.19.52.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786517687384214-19665-0/krb5kdc/kudu_127.19.52.67.keytab.
I20260812 06:55:12.585806 19665 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu
/tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786517687384214-19665-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786517687384214-19665-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786517687384214-19665-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786517687384214-19665-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.52.67:0
--local_ip_for_outbound_sockets=127.19.52.67
--webserver_interface=127.19.52.67
--webserver_port=0
--tserver_master_addrs=127.19.52.126:34489
--keytab_file=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786517687384214-19665-0/krb5kdc/kudu_127.19.52.67.keytab
--principal=kudu/127.19.52.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.52.84:36481
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786517687384214-19665-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786517687384214-19665-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786517687384214-19665-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 06:55:12.705590 24198 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:55:12.705905 24198 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:55:12.705986 24198 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 06:55:12.710484 24198 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 06:55:12.710597 24198 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 06:55:12.710695 24198 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.52.67
I20260812 06:55:12.716722 24198 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.52.84:36481
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786517687384214-19665-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786517687384214-19665-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786517687384214-19665-0/krb5kdc/kudu_127.19.52.67.keytab
--principal=kudu/127.19.52.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.52.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786517687384214-19665-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.19.52.67
--webserver_port=0
--tserver_master_addrs=127.19.52.126:34489
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24198
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.52.67
--log_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786517687384214-19665-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 06:15:54 UTC on ccycloud.quasar-rnfzce.root.comops.site
build id 4581989
I20260812 06:55:12.717911 24198 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:55:12.718971 24198 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 06:55:12.729007 24205 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:55:12.729007 24207 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:55:12.728997 24204 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:55:12.729429 24198 server_base.cc:1034] running on GCE node
I20260812 06:55:12.729846 24198 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:55:12.730532 24198 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:55:12.731724 24198 hybrid_clock.cc:648] HybridClock initialized: now 1786517712731701 us; error 34 us; skew 500 ppm
Aug 12 06:55:12 dist-test-slave-65mx krb5kdc[23738](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517712, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.52.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 06:55:12.734819 24198 init.cc:377] Logged in from keytab as kudu/127.19.52.67@KRBTEST.COM (short username kudu)
I20260812 06:55:12.735730 24198 webserver.cc:460] Webserver started at http://127.19.52.67:43843/ using document root <none> and password file <none>
I20260812 06:55:12.736300 24198 fs_manager.cc:359] Metadata directory not provided
I20260812 06:55:12.736361 24198 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:55:12.736752 24198 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 06:55:12.738870 24198 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786517687384214-19665-0/minicluster-data/ts-2/data/instance:
uuid: "ae445e9ecf824ea2851dc4636eb38f32"
format_stamp: "Formatted at 2026-08-12 06:55:12 on dist-test-slave-65mx"
server_key: "307361fb1267e084e89a37184f166b39"
server_key_iv: "b136ad2f9fa058df0bb86ba92bdaf949"
server_key_version: "encryptionkey@0"
I20260812 06:55:12.739405 24198 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786517687384214-19665-0/minicluster-data/ts-2/wal/instance:
uuid: "ae445e9ecf824ea2851dc4636eb38f32"
format_stamp: "Formatted at 2026-08-12 06:55:12 on dist-test-slave-65mx"
server_key: "307361fb1267e084e89a37184f166b39"
server_key_iv: "b136ad2f9fa058df0bb86ba92bdaf949"
server_key_version: "encryptionkey@0"
I20260812 06:55:12.743582 24198 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260812 06:55:12.746320 24216 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 06:55:12.747723 24198 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260812 06:55:12.747864 24198 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786517687384214-19665-0/minicluster-data/ts-2/data,/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786517687384214-19665-0/minicluster-data/ts-2/wal
uuid: "ae445e9ecf824ea2851dc4636eb38f32"
format_stamp: "Formatted at 2026-08-12 06:55:12 on dist-test-slave-65mx"
server_key: "307361fb1267e084e89a37184f166b39"
server_key_iv: "b136ad2f9fa058df0bb86ba92bdaf949"
server_key_version: "encryptionkey@0"
I20260812 06:55:12.747987 24198 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786517687384214-19665-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786517687384214-19665-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786517687384214-19665-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:55:12.760710 24198 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 06:55:12.764312 24198 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 06:55:12.764606 24198 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:55:12.765334 24198 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 06:55:12.766234 24198 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 06:55:12.766284 24198 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 06:55:12.766326 24198 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 06:55:12.766384 24198 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 06:55:12.777688 24198 rpc_server.cc:307] RPC server started. Bound to: 127.19.52.67:36633
I20260812 06:55:12.777722 24345 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.52.67:36633 every 8 connection(s)
I20260812 06:55:12.778723 24198 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786517687384214-19665-0/minicluster-data/ts-2/data/info.pb
I20260812 06:55:12.783929 19665 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu as pid 24198
I20260812 06:55:12.784111 19665 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786517687384214-19665-0/minicluster-data/ts-2/wal/instance
I20260812 06:55:12.784453 19665 external_mini_cluster.cc:1428] Setting key 1a594bd1384dcaaec2b01d32653c4113
Aug 12 06:55:12 dist-test-slave-65mx krb5kdc[23738](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517712, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.52.67@KRBTEST.COM for kudu/127.19.52.126@KRBTEST.COM
I20260812 06:55:12.794536 24192 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:12.780681 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.52.67:50429 (local address 127.19.52.126:34489)
0812 06:55:12.780851 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0812 06:55:12.780855 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 06:55:12.781846 (+ 991us) server_negotiation.cc:407] Connection header received
0812 06:55:12.783360 (+ 1514us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:55:12.783363 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:55:12.783415 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:55:12.783520 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:55:12.785008 (+ 1488us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:12.785735 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:12.787276 (+ 1541us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:12.787527 (+ 251us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:12.787613 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:55:12.790317 (+ 2704us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:55:12.790320 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:55:12.790322 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:55:12.790349 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:55:12.791905 (+ 1556us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:55:12.792654 (+ 749us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:55:12.792657 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:55:12.792659 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:55:12.792751 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:55:12.793169 (+ 418us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:55:12.793173 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:55:12.793174 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:55:12.793383 (+ 209us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:55:12.793539 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0812 06:55:12.794253 (+ 714us) server_negotiation.cc:299] Negotiation successful
0812 06:55:12.794394 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260812 06:55:12.795411 24348 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:12.781141 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.126:34489 (local address 127.19.52.67:50429)
0812 06:55:12.781675 (+ 534us) negotiation.cc:107] Waiting for socket to connect
0812 06:55:12.781724 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0812 06:55:12.783125 (+ 1401us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:55:12.783701 (+ 576us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:55:12.783708 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:55:12.784194 (+ 486us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:55:12.784844 (+ 650us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:55:12.784861 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:55:12.785921 (+ 1060us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:55:12.785925 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:55:12.787156 (+ 1231us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:55:12.787167 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:55:12.787669 (+ 502us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:55:12.787671 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:55:12.787825 (+ 154us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:55:12.788446 (+ 621us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 06:55:12.788464 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:55:12.790146 (+ 1682us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 06:55:12.792053 (+ 1907us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:55:12.792059 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:55:12.792073 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:55:12.792406 (+ 333us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:55:12.792842 (+ 436us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:55:12.792844 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:55:12.792846 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:55:12.793005 (+ 159us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:55:12.793558 (+ 553us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 06:55:12.793572 (+ 14us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 06:55:12.793842 (+ 270us) client_negotiation.cc:769] Sending connection context
0812 06:55:12.794174 (+ 332us) client_negotiation.cc:240] Negotiation successful
0812 06:55:12.794486 (+ 312us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":289,"thread_start_us":127,"threads_started":1}
I20260812 06:55:12.796773 24346 heartbeater.cc:347] Connected to a master server at 127.19.52.126:34489
I20260812 06:55:12.797052 24346 heartbeater.cc:464] Registering TS with master...
I20260812 06:55:12.797740 24346 heartbeater.cc:511] Master 127.19.52.126:34489 requested a full tablet report, sending...
I20260812 06:55:12.799057 23791 ts_manager.cc:194] Registered new tserver with Master: ae445e9ecf824ea2851dc4636eb38f32 (127.19.52.67:36633)
I20260812 06:55:12.799724 23791 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.52.67@KRBTEST.COM'} at 127.19.52.67:50429
I20260812 06:55:12.811820 19665 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 12 06:55:12 dist-test-slave-65mx krb5kdc[23738](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517712, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 12 06:55:12 dist-test-slave-65mx krb5kdc[23738](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517712, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.19.52.65@KRBTEST.COM
I20260812 06:55:12.843351 24355 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:12.832504 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54724 (local address 127.19.52.65:43519)
0812 06:55:12.832932 (+ 428us) server_negotiation.cc:206] Beginning negotiation
0812 06:55:12.832940 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 06:55:12.832969 (+ 29us) server_negotiation.cc:407] Connection header received
0812 06:55:12.833031 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:55:12.833035 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:55:12.833223 (+ 188us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:55:12.833385 (+ 162us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:55:12.834161 (+ 776us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:12.834970 (+ 809us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:12.835889 (+ 919us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:12.836283 (+ 394us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:12.836418 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:55:12.839212 (+ 2794us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:55:12.839224 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:55:12.839239 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:55:12.839274 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:55:12.841328 (+ 2054us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:55:12.841857 (+ 529us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:55:12.841862 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:55:12.841864 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:55:12.841950 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:55:12.842239 (+ 289us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:55:12.842243 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:55:12.842245 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:55:12.842524 (+ 279us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:55:12.842784 (+ 260us) server_negotiation.cc:1035] Waiting for connection context
0812 06:55:12.843002 (+ 218us) server_negotiation.cc:299] Negotiation successful
0812 06:55:12.843176 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":292,"thread_start_us":144,"threads_started":1}
Aug 12 06:55:12 dist-test-slave-65mx krb5kdc[23738](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517712, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.19.52.126@KRBTEST.COM
I20260812 06:55:12.856920 24192 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:12.846901 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46196 (local address 127.19.52.126:34489)
0812 06:55:12.847093 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0812 06:55:12.847099 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 06:55:12.847165 (+ 66us) server_negotiation.cc:407] Connection header received
0812 06:55:12.847345 (+ 180us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:55:12.847348 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:55:12.847401 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:55:12.847489 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:55:12.848120 (+ 631us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:12.848757 (+ 637us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:12.849545 (+ 788us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:12.849776 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:12.849886 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:55:12.852459 (+ 2573us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:55:12.852465 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:55:12.852470 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:55:12.852506 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:55:12.854227 (+ 1721us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:55:12.855014 (+ 787us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:55:12.855019 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:55:12.855023 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:55:12.855123 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:55:12.855607 (+ 484us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:55:12.855610 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:55:12.855612 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:55:12.855893 (+ 281us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:55:12.856065 (+ 172us) server_negotiation.cc:1035] Waiting for connection context
0812 06:55:12.856370 (+ 305us) server_negotiation.cc:299] Negotiation successful
0812 06:55:12.856712 (+ 342us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":105}
W20260812 06:55:12.857612 23791 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46196
I20260812 06:55:12.858619 19665 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu with pid 23869
I20260812 06:55:12.868124 19665 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu with pid 24034
I20260812 06:55:12.876765 19665 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu with pid 24198
I20260812 06:55:12.888664 19665 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu with pid 23756
2026-08-12T06:55:12Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizedSuperuser (3774 ms)
[ RUN ] SecurityITest.TestDisableWebUI
Loading random data
Initializing database '/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786517687384214-19665-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 06:55:12 dist-test-slave-65mx krb5kdc[24361](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 06:55:12 dist-test-slave-65mx krb5kdc[24361](info): set up 2 sockets
Aug 12 06:55:12 dist-test-slave-65mx krb5kdc[24361](info): commencing operation
krb5kdc: starting...
W20260812 06:55:14.059101 19665 mini_kdc.cc:121] Time spent starting KDC: real 1.144s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786517687384214-19665-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786517687384214-19665-0/krb5kdc/test-user.keytab.
Aug 12 06:55:14 dist-test-slave-65mx krb5kdc[24361](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517714, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T06:55:14Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T06:55:14Z Disabled control of system clock
WARNING: no policy specified for kudu/127.19.52.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.52.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.52.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786517687384214-19665-0/krb5kdc/kudu_127.19.52.126.keytab.
Entry for principal kudu/127.19.52.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786517687384214-19665-0/krb5kdc/kudu_127.19.52.126.keytab.
I20260812 06:55:14.205197 19665 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu
/tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786517687384214-19665-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786517687384214-19665-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786517687384214-19665-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786517687384214-19665-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.52.126:33091
--webserver_interface=127.19.52.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786517687384214-19665-0/krb5kdc/kudu_127.19.52.126.keytab
--principal=kudu/127.19.52.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.52.84:35497
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.19.52.126:33091
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786517687384214-19665-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786517687384214-19665-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786517687384214-19665-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 06:55:14.323436 24377 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:55:14.323766 24377 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:55:14.323853 24377 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 06:55:14.327986 24377 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 06:55:14.328096 24377 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 06:55:14.328137 24377 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 06:55:14.328177 24377 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 06:55:14.328210 24377 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 06:55:14.333722 24377 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.52.84:35497
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786517687384214-19665-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786517687384214-19665-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.19.52.126:33091
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786517687384214-19665-0/krb5kdc/kudu_127.19.52.126.keytab
--principal=kudu/127.19.52.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.52.126:33091
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786517687384214-19665-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.19.52.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24377
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786517687384214-19665-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 06:15:54 UTC on ccycloud.quasar-rnfzce.root.comops.site
build id 4581989
I20260812 06:55:14.334904 24377 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:55:14.335999 24377 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 06:55:14.345081 24384 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:55:14.345091 24386 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:55:14.345117 24383 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:55:14.345467 24377 server_base.cc:1034] running on GCE node
I20260812 06:55:14.346191 24377 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:55:14.347342 24377 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:55:14.348524 24377 hybrid_clock.cc:648] HybridClock initialized: now 1786517714348503 us; error 34 us; skew 500 ppm
Aug 12 06:55:14 dist-test-slave-65mx krb5kdc[24361](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517714, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.52.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 06:55:14.358304 24377 init.cc:377] Logged in from keytab as kudu/127.19.52.126@KRBTEST.COM (short username kudu)
I20260812 06:55:14.359316 24377 fs_manager.cc:359] Metadata directory not provided
I20260812 06:55:14.359416 24377 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:55:14.359862 24377 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 06:55:14.362964 24377 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786517687384214-19665-0/minicluster-data/master-0/data/instance:
uuid: "74514905423a472c8d45396e9b29311f"
format_stamp: "Formatted at 2026-08-12 06:55:14 on dist-test-slave-65mx"
server_key: "39655ba7e32d7a1b6e89fca8aa99b51e"
server_key_iv: "d19278f904d5a345acf41942254907cf"
server_key_version: "encryptionkey@0"
I20260812 06:55:14.364004 24377 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786517687384214-19665-0/minicluster-data/master-0/wal/instance:
uuid: "74514905423a472c8d45396e9b29311f"
format_stamp: "Formatted at 2026-08-12 06:55:14 on dist-test-slave-65mx"
server_key: "39655ba7e32d7a1b6e89fca8aa99b51e"
server_key_iv: "d19278f904d5a345acf41942254907cf"
server_key_version: "encryptionkey@0"
I20260812 06:55:14.368696 24377 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.002s
I20260812 06:55:14.371573 24393 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 06:55:14.373278 24377 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 06:55:14.373448 24377 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786517687384214-19665-0/minicluster-data/master-0/data,/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786517687384214-19665-0/minicluster-data/master-0/wal
uuid: "74514905423a472c8d45396e9b29311f"
format_stamp: "Formatted at 2026-08-12 06:55:14 on dist-test-slave-65mx"
server_key: "39655ba7e32d7a1b6e89fca8aa99b51e"
server_key_iv: "d19278f904d5a345acf41942254907cf"
server_key_version: "encryptionkey@0"
I20260812 06:55:14.373616 24377 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786517687384214-19665-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786517687384214-19665-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786517687384214-19665-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:55:14.392339 24377 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 06:55:14.396235 24377 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 06:55:14.396503 24377 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:55:14.405068 24377 rpc_server.cc:307] RPC server started. Bound to: 127.19.52.126:33091
I20260812 06:55:14.405421 24453 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.52.126:33091 every 8 connection(s)
I20260812 06:55:14.405822 24377 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786517687384214-19665-0/minicluster-data/master-0/data/info.pb
I20260812 06:55:14.409086 24455 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 06:55:14.414045 19665 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu as pid 24377
I20260812 06:55:14.414179 19665 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786517687384214-19665-0/minicluster-data/master-0/wal/instance
I20260812 06:55:14.414492 19665 external_mini_cluster.cc:1428] Setting key 134f718dc907503144a3d68280b39f34
I20260812 06:55:14.415910 24455 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 74514905423a472c8d45396e9b29311f: Bootstrap starting.
I20260812 06:55:14.418787 24455 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 74514905423a472c8d45396e9b29311f: Neither blocks nor log segments found. Creating new log.
I20260812 06:55:14.419658 24455 log.cc:824] T 00000000000000000000000000000000 P 74514905423a472c8d45396e9b29311f: Log is configured to *not* fsync() on all Append() calls
I20260812 06:55:14.422158 24455 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 74514905423a472c8d45396e9b29311f: No bootstrap required, opened a new log
Aug 12 06:55:14 dist-test-slave-65mx krb5kdc[24361](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517714, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.19.52.126@KRBTEST.COM
I20260812 06:55:14.426407 24455 raft_consensus.cc:348] T 00000000000000000000000000000000 P 74514905423a472c8d45396e9b29311f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "74514905423a472c8d45396e9b29311f" member_type: VOTER last_known_addr { host: "127.19.52.126" port: 33091 } }
I20260812 06:55:14.426764 24455 raft_consensus.cc:374] T 00000000000000000000000000000000 P 74514905423a472c8d45396e9b29311f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 06:55:14.426855 24455 raft_consensus.cc:729] T 00000000000000000000000000000000 P 74514905423a472c8d45396e9b29311f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 74514905423a472c8d45396e9b29311f, State: Initialized, Role: FOLLOWER
I20260812 06:55:14.427378 24455 consensus_queue.cc:260] T 00000000000000000000000000000000 P 74514905423a472c8d45396e9b29311f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "74514905423a472c8d45396e9b29311f" member_type: VOTER last_known_addr { host: "127.19.52.126" port: 33091 } }
I20260812 06:55:14.427551 24455 raft_consensus.cc:388] T 00000000000000000000000000000000 P 74514905423a472c8d45396e9b29311f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 06:55:14.427647 24455 raft_consensus.cc:482] T 00000000000000000000000000000000 P 74514905423a472c8d45396e9b29311f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 06:55:14.427775 24455 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 74514905423a472c8d45396e9b29311f [term 0 FOLLOWER]: Advancing to term 1
I20260812 06:55:14.428900 24455 raft_consensus.cc:504] T 00000000000000000000000000000000 P 74514905423a472c8d45396e9b29311f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "74514905423a472c8d45396e9b29311f" member_type: VOTER last_known_addr { host: "127.19.52.126" port: 33091 } }
I20260812 06:55:14.429288 24455 leader_election.cc:302] T 00000000000000000000000000000000 P 74514905423a472c8d45396e9b29311f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 74514905423a472c8d45396e9b29311f; no voters:
I20260812 06:55:14.429687 24455 leader_election.cc:288] T 00000000000000000000000000000000 P 74514905423a472c8d45396e9b29311f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 06:55:14.429806 24462 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 74514905423a472c8d45396e9b29311f [term 1 FOLLOWER]: Leader election won for term 1
I20260812 06:55:14.430189 24462 raft_consensus.cc:686] T 00000000000000000000000000000000 P 74514905423a472c8d45396e9b29311f [term 1 LEADER]: Becoming Leader. State: Replica: 74514905423a472c8d45396e9b29311f, State: Running, Role: LEADER
I20260812 06:55:14.430569 24462 consensus_queue.cc:237] T 00000000000000000000000000000000 P 74514905423a472c8d45396e9b29311f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "74514905423a472c8d45396e9b29311f" member_type: VOTER last_known_addr { host: "127.19.52.126" port: 33091 } }
I20260812 06:55:14.431353 24455 sys_catalog.cc:564] T 00000000000000000000000000000000 P 74514905423a472c8d45396e9b29311f [sys.catalog]: configured and running, proceeding with master startup.
I20260812 06:55:14.431465 24460 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:14.416026 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33788 (local address 127.19.52.126:33091)
0812 06:55:14.416509 (+ 483us) server_negotiation.cc:206] Beginning negotiation
0812 06:55:14.416527 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0812 06:55:14.416625 (+ 98us) server_negotiation.cc:407] Connection header received
0812 06:55:14.417360 (+ 735us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:55:14.417380 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:55:14.417733 (+ 353us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:55:14.418041 (+ 308us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:55:14.419109 (+ 1068us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:14.420185 (+ 1076us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:14.421251 (+ 1066us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:14.421978 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:14.422159 (+ 181us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:55:14.424980 (+ 2821us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:55:14.424988 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:55:14.425015 (+ 27us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:55:14.425060 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:55:14.427910 (+ 2850us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:55:14.428442 (+ 532us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:55:14.428448 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:55:14.428454 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:55:14.428666 (+ 212us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:55:14.428978 (+ 312us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:55:14.428982 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:55:14.428984 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:55:14.429501 (+ 517us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:55:14.429736 (+ 235us) server_negotiation.cc:1035] Waiting for connection context
0812 06:55:14.430011 (+ 275us) server_negotiation.cc:299] Negotiation successful
0812 06:55:14.430361 (+ 350us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":318,"thread_start_us":145,"threads_started":1}
I20260812 06:55:14.432830 24464 sys_catalog.cc:455] T 00000000000000000000000000000000 P 74514905423a472c8d45396e9b29311f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "74514905423a472c8d45396e9b29311f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "74514905423a472c8d45396e9b29311f" member_type: VOTER last_known_addr { host: "127.19.52.126" port: 33091 } } }
I20260812 06:55:14.432988 24464 sys_catalog.cc:458] T 00000000000000000000000000000000 P 74514905423a472c8d45396e9b29311f [sys.catalog]: This master's current role is: LEADER
I20260812 06:55:14.433545 24465 sys_catalog.cc:455] T 00000000000000000000000000000000 P 74514905423a472c8d45396e9b29311f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 74514905423a472c8d45396e9b29311f. Latest consensus state: current_term: 1 leader_uuid: "74514905423a472c8d45396e9b29311f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "74514905423a472c8d45396e9b29311f" member_type: VOTER last_known_addr { host: "127.19.52.126" port: 33091 } } }
I20260812 06:55:14.434054 24465 sys_catalog.cc:458] T 00000000000000000000000000000000 P 74514905423a472c8d45396e9b29311f [sys.catalog]: This master's current role is: LEADER
W20260812 06:55:14.436631 24478 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 74514905423a472c8d45396e9b29311f: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 06:55:14.436717 24478 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 06:55:14.436813 24479 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 06:55:14.437580 24479 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 06:55:14.443316 24479 catalog_manager.cc:1347] Generated new cluster ID: b856100e4d4c4a719ef63a2a107bacca
I20260812 06:55:14.443395 24479 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 06:55:14.452756 24479 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 06:55:14.454116 24479 catalog_manager.cc:1504] Loading token signing keys...
I20260812 06:55:14.463155 24479 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 74514905423a472c8d45396e9b29311f: Generated new TSK 0
I20260812 06:55:14.464035 24479 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.19.52.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.52.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.52.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786517687384214-19665-0/krb5kdc/kudu_127.19.52.65.keytab.
Entry for principal kudu/127.19.52.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786517687384214-19665-0/krb5kdc/kudu_127.19.52.65.keytab.
I20260812 06:55:14.500844 19665 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu
/tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786517687384214-19665-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786517687384214-19665-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786517687384214-19665-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786517687384214-19665-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.52.65:0
--local_ip_for_outbound_sockets=127.19.52.65
--webserver_interface=127.19.52.65
--webserver_port=0
--tserver_master_addrs=127.19.52.126:33091
--keytab_file=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786517687384214-19665-0/krb5kdc/kudu_127.19.52.65.keytab
--principal=kudu/127.19.52.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.52.84:35497
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786517687384214-19665-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786517687384214-19665-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786517687384214-19665-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 06:55:14.621170 24486 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:55:14.621461 24486 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:55:14.621510 24486 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 06:55:14.625816 24486 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 06:55:14.625901 24486 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 06:55:14.626016 24486 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.52.65
I20260812 06:55:14.631599 24486 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.52.84:35497
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786517687384214-19665-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786517687384214-19665-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786517687384214-19665-0/krb5kdc/kudu_127.19.52.65.keytab
--principal=kudu/127.19.52.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.52.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786517687384214-19665-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.19.52.65
--webserver_port=0
--tserver_master_addrs=127.19.52.126:33091
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24486
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.52.65
--log_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786517687384214-19665-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 06:15:54 UTC on ccycloud.quasar-rnfzce.root.comops.site
build id 4581989
I20260812 06:55:14.633045 24486 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:55:14.634182 24486 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 06:55:14.644917 24493 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:55:14.644917 24495 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:55:14.645134 24486 server_base.cc:1034] running on GCE node
W20260812 06:55:14.644999 24497 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:55:14.645794 24486 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:55:14.646529 24486 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:55:14.647713 24486 hybrid_clock.cc:648] HybridClock initialized: now 1786517714647691 us; error 33 us; skew 500 ppm
Aug 12 06:55:14 dist-test-slave-65mx krb5kdc[24361](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517714, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.52.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 06:55:14.651168 24486 init.cc:377] Logged in from keytab as kudu/127.19.52.65@KRBTEST.COM (short username kudu)
I20260812 06:55:14.652069 24486 fs_manager.cc:359] Metadata directory not provided
I20260812 06:55:14.652143 24486 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:55:14.652330 24486 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 06:55:14.654631 24486 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786517687384214-19665-0/minicluster-data/ts-0/data/instance:
uuid: "2f9ad47f839c4e8aad1e7e438a94d377"
format_stamp: "Formatted at 2026-08-12 06:55:14 on dist-test-slave-65mx"
server_key: "a5faa5bb378f55d99b5218f5b6c530e0"
server_key_iv: "faa9ac34f5d7a39294b59961ab296c08"
server_key_version: "encryptionkey@0"
I20260812 06:55:14.655223 24486 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786517687384214-19665-0/minicluster-data/ts-0/wal/instance:
uuid: "2f9ad47f839c4e8aad1e7e438a94d377"
format_stamp: "Formatted at 2026-08-12 06:55:14 on dist-test-slave-65mx"
server_key: "a5faa5bb378f55d99b5218f5b6c530e0"
server_key_iv: "faa9ac34f5d7a39294b59961ab296c08"
server_key_version: "encryptionkey@0"
I20260812 06:55:14.660521 24486 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260812 06:55:14.663666 24503 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 06:55:14.665298 24486 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260812 06:55:14.665482 24486 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786517687384214-19665-0/minicluster-data/ts-0/data,/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786517687384214-19665-0/minicluster-data/ts-0/wal
uuid: "2f9ad47f839c4e8aad1e7e438a94d377"
format_stamp: "Formatted at 2026-08-12 06:55:14 on dist-test-slave-65mx"
server_key: "a5faa5bb378f55d99b5218f5b6c530e0"
server_key_iv: "faa9ac34f5d7a39294b59961ab296c08"
server_key_version: "encryptionkey@0"
I20260812 06:55:14.665642 24486 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786517687384214-19665-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786517687384214-19665-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786517687384214-19665-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:55:14.675868 24486 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 06:55:14.679466 24486 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 06:55:14.679689 24486 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:55:14.680303 24486 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 06:55:14.681257 24486 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 06:55:14.681331 24486 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 06:55:14.681406 24486 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 06:55:14.681456 24486 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 06:55:14.691318 24486 rpc_server.cc:307] RPC server started. Bound to: 127.19.52.65:41221
I20260812 06:55:14.691335 24646 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.52.65:41221 every 8 connection(s)
I20260812 06:55:14.692092 24486 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786517687384214-19665-0/minicluster-data/ts-0/data/info.pb
I20260812 06:55:14.699309 19665 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu as pid 24486
I20260812 06:55:14.699450 19665 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786517687384214-19665-0/minicluster-data/ts-0/wal/instance
I20260812 06:55:14.699750 19665 external_mini_cluster.cc:1428] Setting key 8fd08f911da57ff3b17832df9cef1aca
Aug 12 06:55:14 dist-test-slave-65mx krb5kdc[24361](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517714, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.52.65@KRBTEST.COM for kudu/127.19.52.126@KRBTEST.COM
I20260812 06:55:14.706671 24460 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:14.693900 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.52.65:35903 (local address 127.19.52.126:33091)
0812 06:55:14.694049 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0812 06:55:14.694054 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 06:55:14.695278 (+ 1224us) server_negotiation.cc:407] Connection header received
0812 06:55:14.696196 (+ 918us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:55:14.696200 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:55:14.696261 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:55:14.696367 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:55:14.697642 (+ 1275us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:14.698182 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:14.699489 (+ 1307us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:14.699834 (+ 345us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:14.699961 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:55:14.702749 (+ 2788us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:55:14.702759 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:55:14.702765 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:55:14.702800 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:55:14.704435 (+ 1635us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:55:14.705064 (+ 629us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:55:14.705069 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:55:14.705071 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:55:14.705156 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:55:14.705514 (+ 358us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:55:14.705520 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:55:14.705523 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:55:14.705717 (+ 194us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:55:14.705837 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0812 06:55:14.706361 (+ 524us) server_negotiation.cc:299] Negotiation successful
0812 06:55:14.706502 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260812 06:55:14.707584 24650 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:14.694268 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.126:33091 (local address 127.19.52.65:35903)
0812 06:55:14.695128 (+ 860us) negotiation.cc:107] Waiting for socket to connect
0812 06:55:14.695170 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0812 06:55:14.695986 (+ 816us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:55:14.696581 (+ 595us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:55:14.696591 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:55:14.696971 (+ 380us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:55:14.697495 (+ 524us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:55:14.697513 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:55:14.698312 (+ 799us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:55:14.698319 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:55:14.699312 (+ 993us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:55:14.699323 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:55:14.699995 (+ 672us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:55:14.700041 (+ 46us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:55:14.700213 (+ 172us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:55:14.700944 (+ 731us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 06:55:14.700961 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:55:14.702562 (+ 1601us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 06:55:14.704643 (+ 2081us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:55:14.704650 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:55:14.704674 (+ 24us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:55:14.704945 (+ 271us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:55:14.705256 (+ 311us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:55:14.705260 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:55:14.705262 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:55:14.705411 (+ 149us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:55:14.705837 (+ 426us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 06:55:14.705845 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 06:55:14.706117 (+ 272us) client_negotiation.cc:769] Sending connection context
0812 06:55:14.706316 (+ 199us) client_negotiation.cc:240] Negotiation successful
0812 06:55:14.706688 (+ 372us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":306,"thread_start_us":126,"threads_started":1}
I20260812 06:55:14.708986 24648 heartbeater.cc:347] Connected to a master server at 127.19.52.126:33091
I20260812 06:55:14.709276 24648 heartbeater.cc:464] Registering TS with master...
I20260812 06:55:14.709872 24648 heartbeater.cc:511] Master 127.19.52.126:33091 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.19.52.66@KRBTEST.COM; defaulting to no policy
I20260812 06:55:14.711738 24412 ts_manager.cc:194] Registered new tserver with Master: 2f9ad47f839c4e8aad1e7e438a94d377 (127.19.52.65:41221)
I20260812 06:55:14.713073 24412 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.52.65@KRBTEST.COM'} at 127.19.52.65:35903
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.52.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.52.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786517687384214-19665-0/krb5kdc/kudu_127.19.52.66.keytab.
Entry for principal kudu/127.19.52.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786517687384214-19665-0/krb5kdc/kudu_127.19.52.66.keytab.
I20260812 06:55:14.732867 19665 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu
/tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786517687384214-19665-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786517687384214-19665-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786517687384214-19665-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786517687384214-19665-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.52.66:0
--local_ip_for_outbound_sockets=127.19.52.66
--webserver_interface=127.19.52.66
--webserver_port=0
--tserver_master_addrs=127.19.52.126:33091
--keytab_file=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786517687384214-19665-0/krb5kdc/kudu_127.19.52.66.keytab
--principal=kudu/127.19.52.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.52.84:35497
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786517687384214-19665-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786517687384214-19665-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786517687384214-19665-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 06:55:14.852618 24653 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:55:14.852953 24653 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:55:14.853071 24653 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 06:55:14.857317 24653 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 06:55:14.857517 24653 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 06:55:14.857702 24653 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.52.66
I20260812 06:55:14.863972 24653 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.52.84:35497
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786517687384214-19665-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786517687384214-19665-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786517687384214-19665-0/krb5kdc/kudu_127.19.52.66.keytab
--principal=kudu/127.19.52.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.52.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786517687384214-19665-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.19.52.66
--webserver_port=0
--tserver_master_addrs=127.19.52.126:33091
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24653
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.52.66
--log_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786517687384214-19665-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 06:15:54 UTC on ccycloud.quasar-rnfzce.root.comops.site
build id 4581989
I20260812 06:55:14.865456 24653 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:55:14.866669 24653 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 06:55:14.877441 24666 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:55:14.877486 24661 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:55:14.877607 24653 server_base.cc:1034] running on GCE node
W20260812 06:55:14.877480 24662 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:55:14.878182 24653 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:55:14.878949 24653 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:55:14.880177 24653 hybrid_clock.cc:648] HybridClock initialized: now 1786517714880136 us; error 54 us; skew 500 ppm
Aug 12 06:55:14 dist-test-slave-65mx krb5kdc[24361](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517714, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.52.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 06:55:14.883556 24653 init.cc:377] Logged in from keytab as kudu/127.19.52.66@KRBTEST.COM (short username kudu)
I20260812 06:55:14.884459 24653 fs_manager.cc:359] Metadata directory not provided
I20260812 06:55:14.884589 24653 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:55:14.884853 24653 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 06:55:14.887329 24653 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786517687384214-19665-0/minicluster-data/ts-1/data/instance:
uuid: "0a5a5d1125274452b4c3764689c38c68"
format_stamp: "Formatted at 2026-08-12 06:55:14 on dist-test-slave-65mx"
server_key: "50d25b89e1308a942c230e3ab1cd6530"
server_key_iv: "2375806c93e673511867b1c1d0c3933e"
server_key_version: "encryptionkey@0"
I20260812 06:55:14.888015 24653 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786517687384214-19665-0/minicluster-data/ts-1/wal/instance:
uuid: "0a5a5d1125274452b4c3764689c38c68"
format_stamp: "Formatted at 2026-08-12 06:55:14 on dist-test-slave-65mx"
server_key: "50d25b89e1308a942c230e3ab1cd6530"
server_key_iv: "2375806c93e673511867b1c1d0c3933e"
server_key_version: "encryptionkey@0"
I20260812 06:55:14.892238 24653 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.000s sys 0.006s
I20260812 06:55:14.895242 24672 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 06:55:14.896958 24653 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 06:55:14.897169 24653 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786517687384214-19665-0/minicluster-data/ts-1/data,/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786517687384214-19665-0/minicluster-data/ts-1/wal
uuid: "0a5a5d1125274452b4c3764689c38c68"
format_stamp: "Formatted at 2026-08-12 06:55:14 on dist-test-slave-65mx"
server_key: "50d25b89e1308a942c230e3ab1cd6530"
server_key_iv: "2375806c93e673511867b1c1d0c3933e"
server_key_version: "encryptionkey@0"
I20260812 06:55:14.897315 24653 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786517687384214-19665-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786517687384214-19665-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786517687384214-19665-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:55:14.911576 24653 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 06:55:14.915453 24653 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 06:55:14.915704 24653 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:55:14.916352 24653 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 06:55:14.917316 24653 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 06:55:14.917397 24653 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 06:55:14.917472 24653 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 06:55:14.917518 24653 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 06:55:14.928225 24653 rpc_server.cc:307] RPC server started. Bound to: 127.19.52.66:36765
I20260812 06:55:14.928241 24808 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.52.66:36765 every 8 connection(s)
I20260812 06:55:14.929029 24653 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786517687384214-19665-0/minicluster-data/ts-1/data/info.pb
I20260812 06:55:14.931726 19665 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu as pid 24653
I20260812 06:55:14.931865 19665 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786517687384214-19665-0/minicluster-data/ts-1/wal/instance
I20260812 06:55:14.932169 19665 external_mini_cluster.cc:1428] Setting key 7af871a3cb1aa0be060924109be74f1a
Aug 12 06:55:14 dist-test-slave-65mx krb5kdc[24361](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517714, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.52.66@KRBTEST.COM for kudu/127.19.52.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.19.52.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.52.67@KRBTEST.COM" created.
I20260812 06:55:14.948385 24460 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:14.931254 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.52.66:51189 (local address 127.19.52.126:33091)
0812 06:55:14.931479 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0812 06:55:14.931486 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 06:55:14.933721 (+ 2235us) server_negotiation.cc:407] Connection header received
0812 06:55:14.935302 (+ 1581us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:55:14.935305 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:55:14.935355 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:55:14.935456 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:55:14.937420 (+ 1964us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:14.938284 (+ 864us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:14.940318 (+ 2034us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:14.940697 (+ 379us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:14.940791 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:55:14.944189 (+ 3398us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:55:14.944192 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:55:14.944194 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:55:14.944222 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:55:14.945753 (+ 1531us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:55:14.946337 (+ 584us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:55:14.946340 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:55:14.946341 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:55:14.946508 (+ 167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:55:14.946945 (+ 437us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:55:14.946951 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:55:14.946957 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:55:14.947194 (+ 237us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:55:14.947280 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0812 06:55:14.947985 (+ 705us) server_negotiation.cc:299] Negotiation successful
0812 06:55:14.948167 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260812 06:55:14.949276 24811 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:14.932171 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.126:33091 (local address 127.19.52.66:51189)
0812 06:55:14.933490 (+ 1319us) negotiation.cc:107] Waiting for socket to connect
0812 06:55:14.933588 (+ 98us) client_negotiation.cc:174] Beginning negotiation
0812 06:55:14.934985 (+ 1397us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:55:14.935805 (+ 820us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:55:14.935824 (+ 19us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:55:14.936494 (+ 670us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:55:14.937231 (+ 737us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:55:14.937249 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:55:14.938464 (+ 1215us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:55:14.938471 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:55:14.940135 (+ 1664us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:55:14.940151 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:55:14.940880 (+ 729us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:55:14.940885 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:55:14.941142 (+ 257us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:55:14.942008 (+ 866us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 06:55:14.942032 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:55:14.943977 (+ 1945us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 06:55:14.945887 (+ 1910us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:55:14.945892 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:55:14.945907 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:55:14.946242 (+ 335us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:55:14.946617 (+ 375us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:55:14.946620 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:55:14.946621 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:55:14.946824 (+ 203us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:55:14.947298 (+ 474us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 06:55:14.947303 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 06:55:14.947691 (+ 388us) client_negotiation.cc:769] Sending connection context
0812 06:55:14.947923 (+ 232us) client_negotiation.cc:240] Negotiation successful
0812 06:55:14.948266 (+ 343us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":591,"thread_start_us":388,"threads_started":1}
I20260812 06:55:14.950769 24809 heartbeater.cc:347] Connected to a master server at 127.19.52.126:33091
I20260812 06:55:14.951058 24809 heartbeater.cc:464] Registering TS with master...
I20260812 06:55:14.951557 24809 heartbeater.cc:511] Master 127.19.52.126:33091 requested a full tablet report, sending...
I20260812 06:55:14.952829 24412 ts_manager.cc:194] Registered new tserver with Master: 0a5a5d1125274452b4c3764689c38c68 (127.19.52.66:36765)
I20260812 06:55:14.953470 24412 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.52.66@KRBTEST.COM'} at 127.19.52.66:51189
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.52.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786517687384214-19665-0/krb5kdc/kudu_127.19.52.67.keytab.
Entry for principal kudu/127.19.52.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786517687384214-19665-0/krb5kdc/kudu_127.19.52.67.keytab.
I20260812 06:55:14.973141 19665 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu
/tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786517687384214-19665-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786517687384214-19665-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786517687384214-19665-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786517687384214-19665-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.52.67:0
--local_ip_for_outbound_sockets=127.19.52.67
--webserver_interface=127.19.52.67
--webserver_port=0
--tserver_master_addrs=127.19.52.126:33091
--keytab_file=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786517687384214-19665-0/krb5kdc/kudu_127.19.52.67.keytab
--principal=kudu/127.19.52.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.52.84:35497
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786517687384214-19665-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786517687384214-19665-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786517687384214-19665-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 06:55:15.095172 24815 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:55:15.095512 24815 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:55:15.095597 24815 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 06:55:15.099977 24815 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 06:55:15.100090 24815 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 06:55:15.100219 24815 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.52.67
I20260812 06:55:15.105892 24815 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.52.84:35497
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786517687384214-19665-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786517687384214-19665-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786517687384214-19665-0/krb5kdc/kudu_127.19.52.67.keytab
--principal=kudu/127.19.52.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.52.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786517687384214-19665-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.19.52.67
--webserver_port=0
--tserver_master_addrs=127.19.52.126:33091
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24815
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.52.67
--log_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786517687384214-19665-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 06:15:54 UTC on ccycloud.quasar-rnfzce.root.comops.site
build id 4581989
I20260812 06:55:15.106976 24815 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:55:15.107965 24815 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 06:55:15.117708 24820 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:55:15.117702 24821 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:55:15.117702 24824 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:55:15.118109 24815 server_base.cc:1034] running on GCE node
I20260812 06:55:15.118599 24815 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:55:15.119259 24815 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:55:15.120463 24815 hybrid_clock.cc:648] HybridClock initialized: now 1786517715120441 us; error 35 us; skew 500 ppm
Aug 12 06:55:15 dist-test-slave-65mx krb5kdc[24361](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517715, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.52.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 06:55:15.123632 24815 init.cc:377] Logged in from keytab as kudu/127.19.52.67@KRBTEST.COM (short username kudu)
I20260812 06:55:15.124500 24815 fs_manager.cc:359] Metadata directory not provided
I20260812 06:55:15.124630 24815 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:55:15.124892 24815 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 06:55:15.127177 24815 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786517687384214-19665-0/minicluster-data/ts-2/data/instance:
uuid: "297bfcf8844445a9b471f2811bf1b172"
format_stamp: "Formatted at 2026-08-12 06:55:15 on dist-test-slave-65mx"
server_key: "01b73377ba2a8f4403ef0850b9f86b9d"
server_key_iv: "8aac01c83a2c212b9d019e9e42c02862"
server_key_version: "encryptionkey@0"
I20260812 06:55:15.127813 24815 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786517687384214-19665-0/minicluster-data/ts-2/wal/instance:
uuid: "297bfcf8844445a9b471f2811bf1b172"
format_stamp: "Formatted at 2026-08-12 06:55:15 on dist-test-slave-65mx"
server_key: "01b73377ba2a8f4403ef0850b9f86b9d"
server_key_iv: "8aac01c83a2c212b9d019e9e42c02862"
server_key_version: "encryptionkey@0"
I20260812 06:55:15.132005 24815 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.004s
I20260812 06:55:15.134780 24831 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 06:55:15.136072 24815 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260812 06:55:15.136215 24815 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786517687384214-19665-0/minicluster-data/ts-2/data,/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786517687384214-19665-0/minicluster-data/ts-2/wal
uuid: "297bfcf8844445a9b471f2811bf1b172"
format_stamp: "Formatted at 2026-08-12 06:55:15 on dist-test-slave-65mx"
server_key: "01b73377ba2a8f4403ef0850b9f86b9d"
server_key_iv: "8aac01c83a2c212b9d019e9e42c02862"
server_key_version: "encryptionkey@0"
I20260812 06:55:15.136337 24815 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786517687384214-19665-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786517687384214-19665-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786517687384214-19665-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:55:15.161096 24815 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 06:55:15.164966 24815 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 06:55:15.165210 24815 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:55:15.166014 24815 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 06:55:15.166998 24815 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 06:55:15.167078 24815 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 06:55:15.167160 24815 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 06:55:15.167212 24815 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 06:55:15.177564 24815 rpc_server.cc:307] RPC server started. Bound to: 127.19.52.67:38677
I20260812 06:55:15.177619 24968 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.52.67:38677 every 8 connection(s)
I20260812 06:55:15.178232 24815 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786517687384214-19665-0/minicluster-data/ts-2/data/info.pb
I20260812 06:55:15.182101 19665 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu as pid 24815
I20260812 06:55:15.182243 19665 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786517687384214-19665-0/minicluster-data/ts-2/wal/instance
I20260812 06:55:15.182554 19665 external_mini_cluster.cc:1428] Setting key 2b9d195d9000a56e29c5227a93d241b7
Aug 12 06:55:15 dist-test-slave-65mx krb5kdc[24361](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517715, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.52.67@KRBTEST.COM for kudu/127.19.52.126@KRBTEST.COM
I20260812 06:55:15.192796 24460 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:15.180163 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.52.67:58029 (local address 127.19.52.126:33091)
0812 06:55:15.180331 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0812 06:55:15.180336 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 06:55:15.181666 (+ 1330us) server_negotiation.cc:407] Connection header received
0812 06:55:15.182856 (+ 1190us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:55:15.182858 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:55:15.182936 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:55:15.183038 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:55:15.184600 (+ 1562us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:15.185203 (+ 603us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:15.186373 (+ 1170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:15.186631 (+ 258us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:15.186717 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:55:15.189354 (+ 2637us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:55:15.189358 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:55:15.189359 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:55:15.189384 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:55:15.190723 (+ 1339us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:55:15.191243 (+ 520us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:55:15.191246 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:55:15.191247 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:55:15.191327 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:55:15.191656 (+ 329us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:55:15.191658 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:55:15.191659 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:55:15.191834 (+ 175us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:55:15.191914 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0812 06:55:15.192469 (+ 555us) server_negotiation.cc:299] Negotiation successful
0812 06:55:15.192666 (+ 197us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260812 06:55:15.193626 24971 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:15.180624 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.126:33091 (local address 127.19.52.67:58029)
0812 06:55:15.181515 (+ 891us) negotiation.cc:107] Waiting for socket to connect
0812 06:55:15.181558 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0812 06:55:15.182630 (+ 1072us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:55:15.183233 (+ 603us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:55:15.183242 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:55:15.183763 (+ 521us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:55:15.184411 (+ 648us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:55:15.184426 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:55:15.185330 (+ 904us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:55:15.185333 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:55:15.186260 (+ 927us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:55:15.186269 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:55:15.186788 (+ 519us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:55:15.186791 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:55:15.186969 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:55:15.187629 (+ 660us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 06:55:15.187645 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:55:15.189180 (+ 1535us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 06:55:15.190833 (+ 1653us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:55:15.190837 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:55:15.190850 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:55:15.191141 (+ 291us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:55:15.191419 (+ 278us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:55:15.191422 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:55:15.191423 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:55:15.191574 (+ 151us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:55:15.191966 (+ 392us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 06:55:15.191972 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 06:55:15.192217 (+ 245us) client_negotiation.cc:769] Sending connection context
0812 06:55:15.192391 (+ 174us) client_negotiation.cc:240] Negotiation successful
0812 06:55:15.192757 (+ 366us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":351,"thread_start_us":115,"threads_started":1}
I20260812 06:55:15.194911 24969 heartbeater.cc:347] Connected to a master server at 127.19.52.126:33091
I20260812 06:55:15.195174 24969 heartbeater.cc:464] Registering TS with master...
I20260812 06:55:15.195752 24969 heartbeater.cc:511] Master 127.19.52.126:33091 requested a full tablet report, sending...
I20260812 06:55:15.196841 24412 ts_manager.cc:194] Registered new tserver with Master: 297bfcf8844445a9b471f2811bf1b172 (127.19.52.67:38677)
I20260812 06:55:15.197450 24412 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.52.67@KRBTEST.COM'} at 127.19.52.67:58029
I20260812 06:55:15.209023 19665 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 06:55:15.219074 24460 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:15.211128 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33800 (local address 127.19.52.126:33091)
0812 06:55:15.211271 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0812 06:55:15.211275 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 06:55:15.211392 (+ 117us) server_negotiation.cc:407] Connection header received
0812 06:55:15.211469 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:55:15.211471 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:55:15.211526 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:55:15.211613 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:55:15.212480 (+ 867us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:15.213048 (+ 568us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:15.214114 (+ 1066us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:15.214351 (+ 237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:15.214459 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:55:15.215849 (+ 1390us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:55:15.215852 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:55:15.215854 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:55:15.215879 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:55:15.217480 (+ 1601us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:55:15.217860 (+ 380us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:55:15.217863 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:55:15.217864 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:55:15.217983 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:55:15.218238 (+ 255us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:55:15.218240 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:55:15.218241 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:55:15.218482 (+ 241us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:55:15.218595 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0812 06:55:15.218817 (+ 222us) server_negotiation.cc:299] Negotiation successful
0812 06:55:15.218910 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260812 06:55:15.222402 24412 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33800:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 06:55:15.225490 24412 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 06:55:15.240767 24983 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:15.234758 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49006 (local address 127.19.52.66:36765)
0812 06:55:15.235531 (+ 773us) server_negotiation.cc:206] Beginning negotiation
0812 06:55:15.235543 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0812 06:55:15.235567 (+ 24us) server_negotiation.cc:407] Connection header received
0812 06:55:15.235646 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:55:15.235651 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:55:15.235872 (+ 221us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:55:15.236041 (+ 169us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 06:55:15.236427 (+ 386us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:15.237429 (+ 1002us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:15.239239 (+ 1810us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:15.240059 (+ 820us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:15.240196 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:55:15.240431 (+ 235us) server_negotiation.cc:1035] Waiting for connection context
0812 06:55:15.240484 (+ 53us) server_negotiation.cc:299] Negotiation successful
0812 06:55:15.240596 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":654,"thread_start_us":133,"threads_started":1}
I20260812 06:55:15.240764 24982 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:15.234643 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.66:36765 (local address 127.0.0.1:49006)
0812 06:55:15.235046 (+ 403us) negotiation.cc:107] Waiting for socket to connect
0812 06:55:15.235094 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0812 06:55:15.235229 (+ 135us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:55:15.236061 (+ 832us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:55:15.236068 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:55:15.236092 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 06:55:15.236261 (+ 169us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:55:15.236270 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:55:15.237556 (+ 1286us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:55:15.237562 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:55:15.239089 (+ 1527us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:55:15.239102 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:55:15.240193 (+ 1091us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:55:15.240197 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:55:15.240312 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:55:15.240353 (+ 41us) client_negotiation.cc:769] Sending connection context
0812 06:55:15.240455 (+ 102us) client_negotiation.cc:240] Negotiation successful
0812 06:55:15.240599 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":234,"mutex_wait_us":62,"spinlock_wait_cycles":2944,"thread_start_us":138,"threads_started":1}
I20260812 06:55:15.241179 24981 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:15.234653 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.65:41221 (local address 127.0.0.1:51266)
0812 06:55:15.235921 (+ 1268us) negotiation.cc:107] Waiting for socket to connect
0812 06:55:15.235939 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0812 06:55:15.236013 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:55:15.237008 (+ 995us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:55:15.237011 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:55:15.237030 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 06:55:15.237142 (+ 112us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:55:15.237149 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:55:15.238423 (+ 1274us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:55:15.238429 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:55:15.239417 (+ 988us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:55:15.239428 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:55:15.240764 (+ 1336us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:55:15.240768 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:55:15.240868 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:55:15.240894 (+ 26us) client_negotiation.cc:769] Sending connection context
0812 06:55:15.240950 (+ 56us) client_negotiation.cc:240] Negotiation successful
0812 06:55:15.241034 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1197,"mutex_wait_us":3,"thread_start_us":99,"threads_started":1}
I20260812 06:55:15.241868 24985 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:15.235580 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51266 (local address 127.19.52.65:41221)
0812 06:55:15.236470 (+ 890us) server_negotiation.cc:206] Beginning negotiation
0812 06:55:15.236475 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 06:55:15.236495 (+ 20us) server_negotiation.cc:407] Connection header received
0812 06:55:15.236631 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:55:15.236639 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:55:15.236840 (+ 201us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:55:15.237017 (+ 177us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 06:55:15.237273 (+ 256us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:15.238299 (+ 1026us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:15.239573 (+ 1274us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:15.240622 (+ 1049us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:15.241384 (+ 762us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:55:15.241536 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0812 06:55:15.241598 (+ 62us) server_negotiation.cc:299] Negotiation successful
0812 06:55:15.241712 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":771,"thread_start_us":117,"threads_started":1}
I20260812 06:55:15.242198 24984 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:15.235458 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.67:38677 (local address 127.0.0.1:55244)
0812 06:55:15.236200 (+ 742us) negotiation.cc:107] Waiting for socket to connect
0812 06:55:15.236214 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 06:55:15.236298 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:55:15.237515 (+ 1217us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:55:15.237518 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:55:15.237540 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 06:55:15.237687 (+ 147us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:55:15.237694 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:55:15.239118 (+ 1424us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:55:15.239122 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:55:15.240476 (+ 1354us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:55:15.240489 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:55:15.241722 (+ 1233us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:55:15.241725 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:55:15.241807 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:55:15.241830 (+ 23us) client_negotiation.cc:769] Sending connection context
0812 06:55:15.241900 (+ 70us) client_negotiation.cc:240] Negotiation successful
0812 06:55:15.242069 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":685,"mutex_wait_us":5,"thread_start_us":104,"threads_started":1}
I20260812 06:55:15.242239 24986 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:15.235628 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55244 (local address 127.19.52.67:38677)
0812 06:55:15.236789 (+ 1161us) server_negotiation.cc:206] Beginning negotiation
0812 06:55:15.236801 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0812 06:55:15.236825 (+ 24us) server_negotiation.cc:407] Connection header received
0812 06:55:15.236920 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:55:15.236929 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:55:15.237125 (+ 196us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:55:15.237309 (+ 184us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 06:55:15.237826 (+ 517us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:15.238964 (+ 1138us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:15.240689 (+ 1725us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:15.241589 (+ 900us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:15.241710 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:55:15.241844 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0812 06:55:15.241957 (+ 113us) server_negotiation.cc:299] Negotiation successful
0812 06:55:15.242070 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1059,"thread_start_us":129,"threads_started":1}
I20260812 06:55:15.243965 24726 tablet_service.cc:1467] Processing CreateTablet for tablet 69c7a1839b354e32b98173aefc54f328 (DEFAULT_TABLE table=test-table [id=5b83398c54c64ea5be34b5bd623717af]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 06:55:15.245111 24726 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 69c7a1839b354e32b98173aefc54f328. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 06:55:15.247128 24563 tablet_service.cc:1467] Processing CreateTablet for tablet 69c7a1839b354e32b98173aefc54f328 (DEFAULT_TABLE table=test-table [id=5b83398c54c64ea5be34b5bd623717af]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 06:55:15.247125 24893 tablet_service.cc:1467] Processing CreateTablet for tablet 69c7a1839b354e32b98173aefc54f328 (DEFAULT_TABLE table=test-table [id=5b83398c54c64ea5be34b5bd623717af]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 06:55:15.247991 24563 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 69c7a1839b354e32b98173aefc54f328. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 06:55:15.248004 24893 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 69c7a1839b354e32b98173aefc54f328. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 06:55:15.255031 24987 tablet_bootstrap.cc:492] T 69c7a1839b354e32b98173aefc54f328 P 2f9ad47f839c4e8aad1e7e438a94d377: Bootstrap starting.
I20260812 06:55:15.256011 24989 tablet_bootstrap.cc:492] T 69c7a1839b354e32b98173aefc54f328 P 0a5a5d1125274452b4c3764689c38c68: Bootstrap starting.
I20260812 06:55:15.256835 24990 tablet_bootstrap.cc:492] T 69c7a1839b354e32b98173aefc54f328 P 297bfcf8844445a9b471f2811bf1b172: Bootstrap starting.
I20260812 06:55:15.258203 24987 tablet_bootstrap.cc:654] T 69c7a1839b354e32b98173aefc54f328 P 2f9ad47f839c4e8aad1e7e438a94d377: Neither blocks nor log segments found. Creating new log.
I20260812 06:55:15.258737 24990 tablet_bootstrap.cc:654] T 69c7a1839b354e32b98173aefc54f328 P 297bfcf8844445a9b471f2811bf1b172: Neither blocks nor log segments found. Creating new log.
I20260812 06:55:15.259016 24989 tablet_bootstrap.cc:654] T 69c7a1839b354e32b98173aefc54f328 P 0a5a5d1125274452b4c3764689c38c68: Neither blocks nor log segments found. Creating new log.
I20260812 06:55:15.259469 24987 log.cc:824] T 69c7a1839b354e32b98173aefc54f328 P 2f9ad47f839c4e8aad1e7e438a94d377: Log is configured to *not* fsync() on all Append() calls
I20260812 06:55:15.259999 24990 log.cc:824] T 69c7a1839b354e32b98173aefc54f328 P 297bfcf8844445a9b471f2811bf1b172: Log is configured to *not* fsync() on all Append() calls
I20260812 06:55:15.260263 24989 log.cc:824] T 69c7a1839b354e32b98173aefc54f328 P 0a5a5d1125274452b4c3764689c38c68: Log is configured to *not* fsync() on all Append() calls
I20260812 06:55:15.262216 24987 tablet_bootstrap.cc:492] T 69c7a1839b354e32b98173aefc54f328 P 2f9ad47f839c4e8aad1e7e438a94d377: No bootstrap required, opened a new log
I20260812 06:55:15.262471 24990 tablet_bootstrap.cc:492] T 69c7a1839b354e32b98173aefc54f328 P 297bfcf8844445a9b471f2811bf1b172: No bootstrap required, opened a new log
I20260812 06:55:15.262473 24987 ts_tablet_manager.cc:1397] T 69c7a1839b354e32b98173aefc54f328 P 2f9ad47f839c4e8aad1e7e438a94d377: Time spent bootstrapping tablet: real 0.008s user 0.005s sys 0.000s
I20260812 06:55:15.262609 24989 tablet_bootstrap.cc:492] T 69c7a1839b354e32b98173aefc54f328 P 0a5a5d1125274452b4c3764689c38c68: No bootstrap required, opened a new log
I20260812 06:55:15.262689 24990 ts_tablet_manager.cc:1397] T 69c7a1839b354e32b98173aefc54f328 P 297bfcf8844445a9b471f2811bf1b172: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.004s
I20260812 06:55:15.262802 24989 ts_tablet_manager.cc:1397] T 69c7a1839b354e32b98173aefc54f328 P 0a5a5d1125274452b4c3764689c38c68: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260812 06:55:15.266142 24990 raft_consensus.cc:348] T 69c7a1839b354e32b98173aefc54f328 P 297bfcf8844445a9b471f2811bf1b172 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a5a5d1125274452b4c3764689c38c68" member_type: VOTER last_known_addr { host: "127.19.52.66" port: 36765 } } peers { permanent_uuid: "2f9ad47f839c4e8aad1e7e438a94d377" member_type: VOTER last_known_addr { host: "127.19.52.65" port: 41221 } } peers { permanent_uuid: "297bfcf8844445a9b471f2811bf1b172" member_type: VOTER last_known_addr { host: "127.19.52.67" port: 38677 } }
I20260812 06:55:15.266463 24990 raft_consensus.cc:374] T 69c7a1839b354e32b98173aefc54f328 P 297bfcf8844445a9b471f2811bf1b172 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 06:55:15.266553 24990 raft_consensus.cc:729] T 69c7a1839b354e32b98173aefc54f328 P 297bfcf8844445a9b471f2811bf1b172 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 297bfcf8844445a9b471f2811bf1b172, State: Initialized, Role: FOLLOWER
I20260812 06:55:15.266788 24989 raft_consensus.cc:348] T 69c7a1839b354e32b98173aefc54f328 P 0a5a5d1125274452b4c3764689c38c68 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a5a5d1125274452b4c3764689c38c68" member_type: VOTER last_known_addr { host: "127.19.52.66" port: 36765 } } peers { permanent_uuid: "2f9ad47f839c4e8aad1e7e438a94d377" member_type: VOTER last_known_addr { host: "127.19.52.65" port: 41221 } } peers { permanent_uuid: "297bfcf8844445a9b471f2811bf1b172" member_type: VOTER last_known_addr { host: "127.19.52.67" port: 38677 } }
I20260812 06:55:15.267102 24989 raft_consensus.cc:374] T 69c7a1839b354e32b98173aefc54f328 P 0a5a5d1125274452b4c3764689c38c68 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 06:55:15.267108 24990 consensus_queue.cc:260] T 69c7a1839b354e32b98173aefc54f328 P 297bfcf8844445a9b471f2811bf1b172 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a5a5d1125274452b4c3764689c38c68" member_type: VOTER last_known_addr { host: "127.19.52.66" port: 36765 } } peers { permanent_uuid: "2f9ad47f839c4e8aad1e7e438a94d377" member_type: VOTER last_known_addr { host: "127.19.52.65" port: 41221 } } peers { permanent_uuid: "297bfcf8844445a9b471f2811bf1b172" member_type: VOTER last_known_addr { host: "127.19.52.67" port: 38677 } }
I20260812 06:55:15.267215 24989 raft_consensus.cc:729] T 69c7a1839b354e32b98173aefc54f328 P 0a5a5d1125274452b4c3764689c38c68 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0a5a5d1125274452b4c3764689c38c68, State: Initialized, Role: FOLLOWER
I20260812 06:55:15.267307 24987 raft_consensus.cc:348] T 69c7a1839b354e32b98173aefc54f328 P 2f9ad47f839c4e8aad1e7e438a94d377 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a5a5d1125274452b4c3764689c38c68" member_type: VOTER last_known_addr { host: "127.19.52.66" port: 36765 } } peers { permanent_uuid: "2f9ad47f839c4e8aad1e7e438a94d377" member_type: VOTER last_known_addr { host: "127.19.52.65" port: 41221 } } peers { permanent_uuid: "297bfcf8844445a9b471f2811bf1b172" member_type: VOTER last_known_addr { host: "127.19.52.67" port: 38677 } }
I20260812 06:55:15.267642 24987 raft_consensus.cc:374] T 69c7a1839b354e32b98173aefc54f328 P 2f9ad47f839c4e8aad1e7e438a94d377 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 06:55:15.267663 24989 consensus_queue.cc:260] T 69c7a1839b354e32b98173aefc54f328 P 0a5a5d1125274452b4c3764689c38c68 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a5a5d1125274452b4c3764689c38c68" member_type: VOTER last_known_addr { host: "127.19.52.66" port: 36765 } } peers { permanent_uuid: "2f9ad47f839c4e8aad1e7e438a94d377" member_type: VOTER last_known_addr { host: "127.19.52.65" port: 41221 } } peers { permanent_uuid: "297bfcf8844445a9b471f2811bf1b172" member_type: VOTER last_known_addr { host: "127.19.52.67" port: 38677 } }
I20260812 06:55:15.267745 24987 raft_consensus.cc:729] T 69c7a1839b354e32b98173aefc54f328 P 2f9ad47f839c4e8aad1e7e438a94d377 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2f9ad47f839c4e8aad1e7e438a94d377, State: Initialized, Role: FOLLOWER
I20260812 06:55:15.267959 24969 heartbeater.cc:503] Master 127.19.52.126:33091 was elected leader, sending a full tablet report...
I20260812 06:55:15.268334 24990 ts_tablet_manager.cc:1428] T 69c7a1839b354e32b98173aefc54f328 P 297bfcf8844445a9b471f2811bf1b172: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260812 06:55:15.268268 24987 consensus_queue.cc:260] T 69c7a1839b354e32b98173aefc54f328 P 2f9ad47f839c4e8aad1e7e438a94d377 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a5a5d1125274452b4c3764689c38c68" member_type: VOTER last_known_addr { host: "127.19.52.66" port: 36765 } } peers { permanent_uuid: "2f9ad47f839c4e8aad1e7e438a94d377" member_type: VOTER last_known_addr { host: "127.19.52.65" port: 41221 } } peers { permanent_uuid: "297bfcf8844445a9b471f2811bf1b172" member_type: VOTER last_known_addr { host: "127.19.52.67" port: 38677 } }
I20260812 06:55:15.268761 24989 ts_tablet_manager.cc:1428] T 69c7a1839b354e32b98173aefc54f328 P 0a5a5d1125274452b4c3764689c38c68: Time spent starting tablet: real 0.006s user 0.001s sys 0.006s
I20260812 06:55:15.269176 24809 heartbeater.cc:503] Master 127.19.52.126:33091 was elected leader, sending a full tablet report...
I20260812 06:55:15.269423 24987 ts_tablet_manager.cc:1428] T 69c7a1839b354e32b98173aefc54f328 P 2f9ad47f839c4e8aad1e7e438a94d377: Time spent starting tablet: real 0.007s user 0.002s sys 0.006s
I20260812 06:55:15.269834 24648 heartbeater.cc:503] Master 127.19.52.126:33091 was elected leader, sending a full tablet report...
I20260812 06:55:15.270843 24994 raft_consensus.cc:482] T 69c7a1839b354e32b98173aefc54f328 P 297bfcf8844445a9b471f2811bf1b172 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 06:55:15.271126 24994 raft_consensus.cc:504] T 69c7a1839b354e32b98173aefc54f328 P 297bfcf8844445a9b471f2811bf1b172 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a5a5d1125274452b4c3764689c38c68" member_type: VOTER last_known_addr { host: "127.19.52.66" port: 36765 } } peers { permanent_uuid: "2f9ad47f839c4e8aad1e7e438a94d377" member_type: VOTER last_known_addr { host: "127.19.52.65" port: 41221 } } peers { permanent_uuid: "297bfcf8844445a9b471f2811bf1b172" member_type: VOTER last_known_addr { host: "127.19.52.67" port: 38677 } }
I20260812 06:55:15.272683 24994 leader_election.cc:288] T 69c7a1839b354e32b98173aefc54f328 P 297bfcf8844445a9b471f2811bf1b172 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0a5a5d1125274452b4c3764689c38c68 (127.19.52.66:36765), 2f9ad47f839c4e8aad1e7e438a94d377 (127.19.52.65:41221)
I20260812 06:55:15.276783 24983 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:15.273357 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.52.67:60485 (local address 127.19.52.66:36765)
0812 06:55:15.273467 (+ 110us) server_negotiation.cc:206] Beginning negotiation
0812 06:55:15.273472 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 06:55:15.273487 (+ 15us) server_negotiation.cc:407] Connection header received
0812 06:55:15.273545 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:55:15.273549 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:55:15.273601 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:55:15.273670 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 06:55:15.273944 (+ 274us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:15.274705 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:15.275745 (+ 1040us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:15.276168 (+ 423us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:15.276299 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:55:15.276350 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0812 06:55:15.276589 (+ 239us) server_negotiation.cc:299] Negotiation successful
0812 06:55:15.276660 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260812 06:55:15.276783 24971 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:15.273068 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.66:36765 (local address 127.19.52.67:60485)
0812 06:55:15.273334 (+ 266us) negotiation.cc:107] Waiting for socket to connect
0812 06:55:15.273349 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 06:55:15.273451 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:55:15.273705 (+ 254us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:55:15.273708 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:55:15.273724 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 06:55:15.273843 (+ 119us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:55:15.273850 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:55:15.274816 (+ 966us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:55:15.274822 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:55:15.275617 (+ 795us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:55:15.275626 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:55:15.276293 (+ 667us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:55:15.276297 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:55:15.276402 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:55:15.276423 (+ 21us) client_negotiation.cc:769] Sending connection context
0812 06:55:15.276538 (+ 115us) client_negotiation.cc:240] Negotiation successful
0812 06:55:15.276643 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":183}
I20260812 06:55:15.277374 24997 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:15.273151 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.65:41221 (local address 127.19.52.67:49253)
0812 06:55:15.274115 (+ 964us) negotiation.cc:107] Waiting for socket to connect
0812 06:55:15.274129 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 06:55:15.274228 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:55:15.274440 (+ 212us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:55:15.274443 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:55:15.274459 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 06:55:15.274586 (+ 127us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:55:15.274591 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:55:15.275269 (+ 678us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:55:15.275274 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:55:15.276089 (+ 815us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:55:15.276099 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:55:15.276934 (+ 835us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:55:15.276938 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:55:15.277054 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:55:15.277072 (+ 18us) client_negotiation.cc:769] Sending connection context
0812 06:55:15.277158 (+ 86us) client_negotiation.cc:240] Negotiation successful
0812 06:55:15.277227 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":900,"thread_start_us":130,"threads_started":1}
I20260812 06:55:15.277359 24985 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:15.273831 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.52.67:49253 (local address 127.19.52.65:41221)
0812 06:55:15.273974 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0812 06:55:15.273979 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 06:55:15.274208 (+ 229us) server_negotiation.cc:407] Connection header received
0812 06:55:15.274301 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:55:15.274303 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:55:15.274340 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:55:15.274402 (+ 62us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 06:55:15.274683 (+ 281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:15.275152 (+ 469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:15.276215 (+ 1063us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:15.276782 (+ 567us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:15.276918 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:55:15.276971 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0812 06:55:15.277175 (+ 204us) server_negotiation.cc:299] Negotiation successful
0812 06:55:15.277248 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260812 06:55:15.277709 24750 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "69c7a1839b354e32b98173aefc54f328" candidate_uuid: "297bfcf8844445a9b471f2811bf1b172" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0a5a5d1125274452b4c3764689c38c68" is_pre_election: true
I20260812 06:55:15.278048 24750 raft_consensus.cc:2454] T 69c7a1839b354e32b98173aefc54f328 P 0a5a5d1125274452b4c3764689c38c68 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 297bfcf8844445a9b471f2811bf1b172 in term 0.
I20260812 06:55:15.278049 24589 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "69c7a1839b354e32b98173aefc54f328" candidate_uuid: "297bfcf8844445a9b471f2811bf1b172" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2f9ad47f839c4e8aad1e7e438a94d377" is_pre_election: true
I20260812 06:55:15.278311 24589 raft_consensus.cc:2454] T 69c7a1839b354e32b98173aefc54f328 P 2f9ad47f839c4e8aad1e7e438a94d377 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 297bfcf8844445a9b471f2811bf1b172 in term 0.
I20260812 06:55:15.278558 24834 leader_election.cc:302] T 69c7a1839b354e32b98173aefc54f328 P 297bfcf8844445a9b471f2811bf1b172 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a5a5d1125274452b4c3764689c38c68, 297bfcf8844445a9b471f2811bf1b172; no voters:
I20260812 06:55:15.278926 24994 raft_consensus.cc:2781] T 69c7a1839b354e32b98173aefc54f328 P 297bfcf8844445a9b471f2811bf1b172 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 06:55:15.279018 24994 raft_consensus.cc:482] T 69c7a1839b354e32b98173aefc54f328 P 297bfcf8844445a9b471f2811bf1b172 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 06:55:15.279063 24994 raft_consensus.cc:3037] T 69c7a1839b354e32b98173aefc54f328 P 297bfcf8844445a9b471f2811bf1b172 [term 0 FOLLOWER]: Advancing to term 1
I20260812 06:55:15.280202 24994 raft_consensus.cc:504] T 69c7a1839b354e32b98173aefc54f328 P 297bfcf8844445a9b471f2811bf1b172 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a5a5d1125274452b4c3764689c38c68" member_type: VOTER last_known_addr { host: "127.19.52.66" port: 36765 } } peers { permanent_uuid: "2f9ad47f839c4e8aad1e7e438a94d377" member_type: VOTER last_known_addr { host: "127.19.52.65" port: 41221 } } peers { permanent_uuid: "297bfcf8844445a9b471f2811bf1b172" member_type: VOTER last_known_addr { host: "127.19.52.67" port: 38677 } }
I20260812 06:55:15.280524 24994 leader_election.cc:288] T 69c7a1839b354e32b98173aefc54f328 P 297bfcf8844445a9b471f2811bf1b172 [CANDIDATE]: Term 1 election: Requested vote from peers 0a5a5d1125274452b4c3764689c38c68 (127.19.52.66:36765), 2f9ad47f839c4e8aad1e7e438a94d377 (127.19.52.65:41221)
I20260812 06:55:15.280954 24589 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "69c7a1839b354e32b98173aefc54f328" candidate_uuid: "297bfcf8844445a9b471f2811bf1b172" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2f9ad47f839c4e8aad1e7e438a94d377"
I20260812 06:55:15.280954 24750 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "69c7a1839b354e32b98173aefc54f328" candidate_uuid: "297bfcf8844445a9b471f2811bf1b172" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0a5a5d1125274452b4c3764689c38c68"
I20260812 06:55:15.281092 24589 raft_consensus.cc:3037] T 69c7a1839b354e32b98173aefc54f328 P 2f9ad47f839c4e8aad1e7e438a94d377 [term 0 FOLLOWER]: Advancing to term 1
I20260812 06:55:15.281133 24750 raft_consensus.cc:3037] T 69c7a1839b354e32b98173aefc54f328 P 0a5a5d1125274452b4c3764689c38c68 [term 0 FOLLOWER]: Advancing to term 1
I20260812 06:55:15.282459 24750 raft_consensus.cc:2454] T 69c7a1839b354e32b98173aefc54f328 P 0a5a5d1125274452b4c3764689c38c68 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 297bfcf8844445a9b471f2811bf1b172 in term 1.
I20260812 06:55:15.282459 24589 raft_consensus.cc:2454] T 69c7a1839b354e32b98173aefc54f328 P 2f9ad47f839c4e8aad1e7e438a94d377 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 297bfcf8844445a9b471f2811bf1b172 in term 1.
I20260812 06:55:15.282828 24835 leader_election.cc:302] T 69c7a1839b354e32b98173aefc54f328 P 297bfcf8844445a9b471f2811bf1b172 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 297bfcf8844445a9b471f2811bf1b172, 2f9ad47f839c4e8aad1e7e438a94d377; no voters:
I20260812 06:55:15.283102 24994 raft_consensus.cc:2781] T 69c7a1839b354e32b98173aefc54f328 P 297bfcf8844445a9b471f2811bf1b172 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 06:55:15.283389 24994 raft_consensus.cc:686] T 69c7a1839b354e32b98173aefc54f328 P 297bfcf8844445a9b471f2811bf1b172 [term 1 LEADER]: Becoming Leader. State: Replica: 297bfcf8844445a9b471f2811bf1b172, State: Running, Role: LEADER
I20260812 06:55:15.283795 24994 consensus_queue.cc:237] T 69c7a1839b354e32b98173aefc54f328 P 297bfcf8844445a9b471f2811bf1b172 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a5a5d1125274452b4c3764689c38c68" member_type: VOTER last_known_addr { host: "127.19.52.66" port: 36765 } } peers { permanent_uuid: "2f9ad47f839c4e8aad1e7e438a94d377" member_type: VOTER last_known_addr { host: "127.19.52.65" port: 41221 } } peers { permanent_uuid: "297bfcf8844445a9b471f2811bf1b172" member_type: VOTER last_known_addr { host: "127.19.52.67" port: 38677 } }
I20260812 06:55:15.287156 24411 catalog_manager.cc:5495] T 69c7a1839b354e32b98173aefc54f328 P 297bfcf8844445a9b471f2811bf1b172 reported cstate change: term changed from 0 to 1, leader changed from <none> to 297bfcf8844445a9b471f2811bf1b172 (127.19.52.67). New cstate: current_term: 1 leader_uuid: "297bfcf8844445a9b471f2811bf1b172" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a5a5d1125274452b4c3764689c38c68" member_type: VOTER last_known_addr { host: "127.19.52.66" port: 36765 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2f9ad47f839c4e8aad1e7e438a94d377" member_type: VOTER last_known_addr { host: "127.19.52.65" port: 41221 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "297bfcf8844445a9b471f2811bf1b172" member_type: VOTER last_known_addr { host: "127.19.52.67" port: 38677 } health_report { overall_health: HEALTHY } } }
I20260812 06:55:15.311757 24986 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:15.308122 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55256 (local address 127.19.52.67:38677)
0812 06:55:15.308283 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0812 06:55:15.308288 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 06:55:15.308305 (+ 17us) server_negotiation.cc:407] Connection header received
0812 06:55:15.308402 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:55:15.308405 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:55:15.308459 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:55:15.308608 (+ 149us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 06:55:15.308864 (+ 256us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:15.309378 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:15.310265 (+ 887us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:15.310485 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:15.310611 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:55:15.310927 (+ 316us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 06:55:15.311346 (+ 419us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 06:55:15.311481 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0812 06:55:15.311548 (+ 67us) server_negotiation.cc:299] Negotiation successful
0812 06:55:15.311608 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260812 06:55:15.317710 24589 raft_consensus.cc:1261] T 69c7a1839b354e32b98173aefc54f328 P 2f9ad47f839c4e8aad1e7e438a94d377 [term 1 FOLLOWER]: Refusing update from remote peer 297bfcf8844445a9b471f2811bf1b172: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 06:55:15.317710 24750 raft_consensus.cc:1261] T 69c7a1839b354e32b98173aefc54f328 P 0a5a5d1125274452b4c3764689c38c68 [term 1 FOLLOWER]: Refusing update from remote peer 297bfcf8844445a9b471f2811bf1b172: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 06:55:15.318461 24994 consensus_queue.cc:1035] T 69c7a1839b354e32b98173aefc54f328 P 297bfcf8844445a9b471f2811bf1b172 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a5a5d1125274452b4c3764689c38c68" member_type: VOTER last_known_addr { host: "127.19.52.66" port: 36765 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 06:55:15.318675 24998 consensus_queue.cc:1035] T 69c7a1839b354e32b98173aefc54f328 P 297bfcf8844445a9b471f2811bf1b172 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2f9ad47f839c4e8aad1e7e438a94d377" member_type: VOTER last_known_addr { host: "127.19.52.65" port: 41221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 06:55:15.326433 25002 mvcc.cc:205] Tried to move back new op lower bound from 7317576561933647872 to 7317576561805340672. Current Snapshot: MvccSnapshot[applied={T|T < 7317576561933647872}]
I20260812 06:55:15.335019 24411 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33800:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 06:55:15.335230 24411 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33800:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 06:55:15.337878 24411 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 74514905423a472c8d45396e9b29311f: Sending DeleteTablet for 3 replicas of tablet 69c7a1839b354e32b98173aefc54f328
I20260812 06:55:15.338713 24726 tablet_service.cc:1514] Processing DeleteTablet for tablet 69c7a1839b354e32b98173aefc54f328 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 06:55:15 UTC) from {username='kudu', principal='kudu/127.19.52.126@KRBTEST.COM'} at 127.0.0.1:49006
I20260812 06:55:15.338845 24893 tablet_service.cc:1514] Processing DeleteTablet for tablet 69c7a1839b354e32b98173aefc54f328 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 06:55:15 UTC) from {username='kudu', principal='kudu/127.19.52.126@KRBTEST.COM'} at 127.0.0.1:55244
I20260812 06:55:15.338842 24563 tablet_service.cc:1514] Processing DeleteTablet for tablet 69c7a1839b354e32b98173aefc54f328 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 06:55:15 UTC) from {username='kudu', principal='kudu/127.19.52.126@KRBTEST.COM'} at 127.0.0.1:51266
I20260812 06:55:15.339313 25015 tablet_replica.cc:330] T 69c7a1839b354e32b98173aefc54f328 P 0a5a5d1125274452b4c3764689c38c68: stopping tablet replica
I20260812 06:55:15.339504 25016 tablet_replica.cc:330] T 69c7a1839b354e32b98173aefc54f328 P 2f9ad47f839c4e8aad1e7e438a94d377: stopping tablet replica
I20260812 06:55:15.339577 25013 tablet_replica.cc:330] T 69c7a1839b354e32b98173aefc54f328 P 297bfcf8844445a9b471f2811bf1b172: stopping tablet replica
I20260812 06:55:15.339679 25015 raft_consensus.cc:2229] T 69c7a1839b354e32b98173aefc54f328 P 0a5a5d1125274452b4c3764689c38c68 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 06:55:15.339890 25016 raft_consensus.cc:2229] T 69c7a1839b354e32b98173aefc54f328 P 2f9ad47f839c4e8aad1e7e438a94d377 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 06:55:15.339908 25013 raft_consensus.cc:2229] T 69c7a1839b354e32b98173aefc54f328 P 297bfcf8844445a9b471f2811bf1b172 [term 1 LEADER]: Raft consensus shutting down.
I20260812 06:55:15.339946 19665 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu with pid 24486
I20260812 06:55:15.340102 25015 raft_consensus.cc:2258] T 69c7a1839b354e32b98173aefc54f328 P 0a5a5d1125274452b4c3764689c38c68 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 06:55:15.340319 25016 raft_consensus.cc:2258] T 69c7a1839b354e32b98173aefc54f328 P 2f9ad47f839c4e8aad1e7e438a94d377 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 06:55:15.341574 25013 raft_consensus.cc:2258] T 69c7a1839b354e32b98173aefc54f328 P 297bfcf8844445a9b471f2811bf1b172 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 06:55:15.345209 25015 ts_tablet_manager.cc:1905] T 69c7a1839b354e32b98173aefc54f328 P 0a5a5d1125274452b4c3764689c38c68: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 06:55:15.345415 25013 ts_tablet_manager.cc:1905] T 69c7a1839b354e32b98173aefc54f328 P 297bfcf8844445a9b471f2811bf1b172: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 06:55:15.348256 25013 ts_tablet_manager.cc:1918] T 69c7a1839b354e32b98173aefc54f328 P 297bfcf8844445a9b471f2811bf1b172: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 06:55:15.348410 25013 log.cc:1196] T 69c7a1839b354e32b98173aefc54f328 P 297bfcf8844445a9b471f2811bf1b172: Deleting WAL directory at /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786517687384214-19665-0/minicluster-data/ts-2/wal/wals/69c7a1839b354e32b98173aefc54f328
I20260812 06:55:15.348809 25013 ts_tablet_manager.cc:1939] T 69c7a1839b354e32b98173aefc54f328 P 297bfcf8844445a9b471f2811bf1b172: Deleting consensus metadata
I20260812 06:55:15.349057 25015 ts_tablet_manager.cc:1918] T 69c7a1839b354e32b98173aefc54f328 P 0a5a5d1125274452b4c3764689c38c68: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 06:55:15.349246 25015 log.cc:1196] T 69c7a1839b354e32b98173aefc54f328 P 0a5a5d1125274452b4c3764689c38c68: Deleting WAL directory at /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786517687384214-19665-0/minicluster-data/ts-1/wal/wals/69c7a1839b354e32b98173aefc54f328
I20260812 06:55:15.349664 25015 ts_tablet_manager.cc:1939] T 69c7a1839b354e32b98173aefc54f328 P 0a5a5d1125274452b4c3764689c38c68: Deleting consensus metadata
I20260812 06:55:15.350194 24398 catalog_manager.cc:4841] TS 297bfcf8844445a9b471f2811bf1b172 (127.19.52.67:38677): tablet 69c7a1839b354e32b98173aefc54f328 (table test-table [id=5b83398c54c64ea5be34b5bd623717af]) successfully deleted
I20260812 06:55:15.351035 24395 catalog_manager.cc:4841] TS 0a5a5d1125274452b4c3764689c38c68 (127.19.52.66:36765): tablet 69c7a1839b354e32b98173aefc54f328 (table test-table [id=5b83398c54c64ea5be34b5bd623717af]) successfully deleted
W20260812 06:55:15.352846 24396 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.19.52.65:41221 (error 108)
I20260812 06:55:15.353014 19665 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu with pid 24653
I20260812 06:55:15.356112 24984 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:15.355654 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.65:41221 (local address 127.0.0.1:47220)
0812 06:55:15.355879 (+ 225us) negotiation.cc:107] Waiting for socket to connect
0812 06:55:15.355975 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.52.65:41221: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":125}
W20260812 06:55:15.356410 24396 catalog_manager.cc:4584] TS 2f9ad47f839c4e8aad1e7e438a94d377 (127.19.52.65:41221): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 69c7a1839b354e32b98173aefc54f328: Network error: Client connection negotiation failed: client connection to 127.19.52.65:41221: connect: Connection refused (error 111)
I20260812 06:55:15.364454 19665 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu with pid 24815
I20260812 06:55:15.374018 19665 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu with pid 24377
2026-08-12T06:55:15Z chronyd exiting
[ OK ] SecurityITest.TestDisableWebUI (2484 ms)
[ RUN ] SecurityITest.TestDisableAuthenticationEncryption
2026-08-12T06:55:15Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T06:55:15Z Disabled control of system clock
I20260812 06:55:15.411458 19665 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu
/tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786517687384214-19665-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786517687384214-19665-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786517687384214-19665-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786517687384214-19665-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.52.126:43689
--webserver_interface=127.19.52.126
--webserver_port=0
--builtin_ntp_servers=127.19.52.84:38249
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.19.52.126:43689
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260812 06:55:15.533905 25021 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:55:15.534216 25021 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:55:15.534255 25021 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 06:55:15.538381 25021 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 06:55:15.538446 25021 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 06:55:15.538462 25021 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 06:55:15.538479 25021 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 06:55:15.538496 25021 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 06:55:15.543568 25021 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.52.84:38249
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786517687384214-19665-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786517687384214-19665-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.19.52.126:43689
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.52.126:43689
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786517687384214-19665-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.19.52.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25021
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786517687384214-19665-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 06:15:54 UTC on ccycloud.quasar-rnfzce.root.comops.site
build id 4581989
I20260812 06:55:15.544749 25021 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:55:15.546211 25021 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 06:55:15.556329 25031 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:55:15.556298 25029 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:55:15.556300 25027 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:55:15.556376 25021 server_base.cc:1034] running on GCE node
I20260812 06:55:15.557515 25021 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:55:15.558934 25021 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:55:15.560134 25021 hybrid_clock.cc:648] HybridClock initialized: now 1786517715560116 us; error 39 us; skew 500 ppm
I20260812 06:55:15.562839 25021 webserver.cc:460] Webserver started at http://127.19.52.126:33399/ using document root <none> and password file <none>
I20260812 06:55:15.563573 25021 fs_manager.cc:359] Metadata directory not provided
I20260812 06:55:15.563658 25021 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:55:15.563993 25021 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 06:55:15.566582 25021 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786517687384214-19665-0/minicluster-data/master-0/data/instance:
uuid: "b7e4ad509c3f4c19a716829db7c9ed79"
format_stamp: "Formatted at 2026-08-12 06:55:15 on dist-test-slave-65mx"
server_key: "b6d3c930ca07c4ef76135d1b8fa414ef"
server_key_iv: "78116200871e83d64b77b96bbaec4410"
server_key_version: "encryptionkey@0"
I20260812 06:55:15.567271 25021 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786517687384214-19665-0/minicluster-data/master-0/wal/instance:
uuid: "b7e4ad509c3f4c19a716829db7c9ed79"
format_stamp: "Formatted at 2026-08-12 06:55:15 on dist-test-slave-65mx"
server_key: "b6d3c930ca07c4ef76135d1b8fa414ef"
server_key_iv: "78116200871e83d64b77b96bbaec4410"
server_key_version: "encryptionkey@0"
I20260812 06:55:15.571610 25021 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.004s
I20260812 06:55:15.574563 25039 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 06:55:15.575955 25021 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260812 06:55:15.576134 25021 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786517687384214-19665-0/minicluster-data/master-0/data,/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786517687384214-19665-0/minicluster-data/master-0/wal
uuid: "b7e4ad509c3f4c19a716829db7c9ed79"
format_stamp: "Formatted at 2026-08-12 06:55:15 on dist-test-slave-65mx"
server_key: "b6d3c930ca07c4ef76135d1b8fa414ef"
server_key_iv: "78116200871e83d64b77b96bbaec4410"
server_key_version: "encryptionkey@0"
I20260812 06:55:15.576269 25021 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786517687384214-19665-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786517687384214-19665-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786517687384214-19665-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:55:15.581219 25021 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 06:55:15.581913 25021 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 06:55:15.582116 25021 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:55:15.589603 25021 rpc_server.cc:307] RPC server started. Bound to: 127.19.52.126:43689
I20260812 06:55:15.589635 25103 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.52.126:43689 every 8 connection(s)
I20260812 06:55:15.590576 25021 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786517687384214-19665-0/minicluster-data/master-0/data/info.pb
I20260812 06:55:15.592104 19665 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu as pid 25021
I20260812 06:55:15.592232 19665 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786517687384214-19665-0/minicluster-data/master-0/wal/instance
I20260812 06:55:15.592581 19665 external_mini_cluster.cc:1428] Setting key 9cf9e31ae02deec55c397731a58e3ec5
I20260812 06:55:15.594609 25104 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 06:55:15.599040 25107 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:15.594114 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36340 (local address 127.19.52.126:43689)
0812 06:55:15.594963 (+ 849us) server_negotiation.cc:206] Beginning negotiation
0812 06:55:15.594990 (+ 27us) server_negotiation.cc:399] Waiting for connection header
0812 06:55:15.595030 (+ 40us) server_negotiation.cc:407] Connection header received
0812 06:55:15.595898 (+ 868us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:55:15.595918 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:55:15.596297 (+ 379us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:55:15.596766 (+ 469us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:55:15.597673 (+ 907us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:55:15.597677 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:55:15.597689 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 06:55:15.597700 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:55:15.597757 (+ 57us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 06:55:15.597773 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:55:15.597936 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0812 06:55:15.598010 (+ 74us) server_negotiation.cc:299] Negotiation successful
0812 06:55:15.598267 (+ 257us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":417,"thread_start_us":182,"threads_started":1}
I20260812 06:55:15.601176 25104 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b7e4ad509c3f4c19a716829db7c9ed79: Bootstrap starting.
I20260812 06:55:15.603443 25104 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b7e4ad509c3f4c19a716829db7c9ed79: Neither blocks nor log segments found. Creating new log.
I20260812 06:55:15.604317 25104 log.cc:824] T 00000000000000000000000000000000 P b7e4ad509c3f4c19a716829db7c9ed79: Log is configured to *not* fsync() on all Append() calls
I20260812 06:55:15.606365 25104 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b7e4ad509c3f4c19a716829db7c9ed79: No bootstrap required, opened a new log
I20260812 06:55:15.609987 25104 raft_consensus.cc:348] T 00000000000000000000000000000000 P b7e4ad509c3f4c19a716829db7c9ed79 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b7e4ad509c3f4c19a716829db7c9ed79" member_type: VOTER last_known_addr { host: "127.19.52.126" port: 43689 } }
I20260812 06:55:15.610260 25104 raft_consensus.cc:374] T 00000000000000000000000000000000 P b7e4ad509c3f4c19a716829db7c9ed79 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 06:55:15.610342 25104 raft_consensus.cc:729] T 00000000000000000000000000000000 P b7e4ad509c3f4c19a716829db7c9ed79 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b7e4ad509c3f4c19a716829db7c9ed79, State: Initialized, Role: FOLLOWER
I20260812 06:55:15.610770 25104 consensus_queue.cc:260] T 00000000000000000000000000000000 P b7e4ad509c3f4c19a716829db7c9ed79 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b7e4ad509c3f4c19a716829db7c9ed79" member_type: VOTER last_known_addr { host: "127.19.52.126" port: 43689 } }
I20260812 06:55:15.610917 25104 raft_consensus.cc:388] T 00000000000000000000000000000000 P b7e4ad509c3f4c19a716829db7c9ed79 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 06:55:15.611006 25104 raft_consensus.cc:482] T 00000000000000000000000000000000 P b7e4ad509c3f4c19a716829db7c9ed79 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 06:55:15.611127 25104 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b7e4ad509c3f4c19a716829db7c9ed79 [term 0 FOLLOWER]: Advancing to term 1
I20260812 06:55:15.612087 25104 raft_consensus.cc:504] T 00000000000000000000000000000000 P b7e4ad509c3f4c19a716829db7c9ed79 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b7e4ad509c3f4c19a716829db7c9ed79" member_type: VOTER last_known_addr { host: "127.19.52.126" port: 43689 } }
I20260812 06:55:15.612424 25104 leader_election.cc:302] T 00000000000000000000000000000000 P b7e4ad509c3f4c19a716829db7c9ed79 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b7e4ad509c3f4c19a716829db7c9ed79; no voters:
I20260812 06:55:15.612764 25104 leader_election.cc:288] T 00000000000000000000000000000000 P b7e4ad509c3f4c19a716829db7c9ed79 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 06:55:15.612905 25109 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b7e4ad509c3f4c19a716829db7c9ed79 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 06:55:15.613209 25109 raft_consensus.cc:686] T 00000000000000000000000000000000 P b7e4ad509c3f4c19a716829db7c9ed79 [term 1 LEADER]: Becoming Leader. State: Replica: b7e4ad509c3f4c19a716829db7c9ed79, State: Running, Role: LEADER
I20260812 06:55:15.613515 25109 consensus_queue.cc:237] T 00000000000000000000000000000000 P b7e4ad509c3f4c19a716829db7c9ed79 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b7e4ad509c3f4c19a716829db7c9ed79" member_type: VOTER last_known_addr { host: "127.19.52.126" port: 43689 } }
I20260812 06:55:15.614030 25104 sys_catalog.cc:564] T 00000000000000000000000000000000 P b7e4ad509c3f4c19a716829db7c9ed79 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 06:55:15.615137 25110 sys_catalog.cc:455] T 00000000000000000000000000000000 P b7e4ad509c3f4c19a716829db7c9ed79 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b7e4ad509c3f4c19a716829db7c9ed79" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b7e4ad509c3f4c19a716829db7c9ed79" member_type: VOTER last_known_addr { host: "127.19.52.126" port: 43689 } } }
I20260812 06:55:15.615267 25110 sys_catalog.cc:458] T 00000000000000000000000000000000 P b7e4ad509c3f4c19a716829db7c9ed79 [sys.catalog]: This master's current role is: LEADER
I20260812 06:55:15.615597 25111 sys_catalog.cc:455] T 00000000000000000000000000000000 P b7e4ad509c3f4c19a716829db7c9ed79 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b7e4ad509c3f4c19a716829db7c9ed79. Latest consensus state: current_term: 1 leader_uuid: "b7e4ad509c3f4c19a716829db7c9ed79" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b7e4ad509c3f4c19a716829db7c9ed79" member_type: VOTER last_known_addr { host: "127.19.52.126" port: 43689 } } }
I20260812 06:55:15.615672 25119 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 06:55:15.616662 25111 sys_catalog.cc:458] T 00000000000000000000000000000000 P b7e4ad509c3f4c19a716829db7c9ed79 [sys.catalog]: This master's current role is: LEADER
I20260812 06:55:15.619086 25119 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 06:55:15.624919 25119 catalog_manager.cc:1347] Generated new cluster ID: 1de7a1e82b16426899334a57023db8f8
I20260812 06:55:15.625027 25119 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 06:55:15.662016 25119 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 06:55:15.662567 25119 catalog_manager.cc:1504] Loading token signing keys...
I20260812 06:55:15.673552 25119 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b7e4ad509c3f4c19a716829db7c9ed79: Generated new TSK 0
I20260812 06:55:15.674525 25119 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 06:55:15.681010 19665 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu
/tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786517687384214-19665-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786517687384214-19665-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786517687384214-19665-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786517687384214-19665-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.52.65:0
--local_ip_for_outbound_sockets=127.19.52.65
--webserver_interface=127.19.52.65
--webserver_port=0
--tserver_master_addrs=127.19.52.126:43689
--builtin_ntp_servers=127.19.52.84:38249
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260812 06:55:15.801853 25129 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:55:15.802150 25129 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:55:15.802191 25129 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 06:55:15.806409 25129 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 06:55:15.806479 25129 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 06:55:15.806563 25129 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.52.65
I20260812 06:55:15.811594 25129 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.52.84:38249
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786517687384214-19665-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786517687384214-19665-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.52.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786517687384214-19665-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.19.52.65
--webserver_port=0
--tserver_master_addrs=127.19.52.126:43689
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25129
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.52.65
--log_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786517687384214-19665-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 06:15:54 UTC on ccycloud.quasar-rnfzce.root.comops.site
build id 4581989
I20260812 06:55:15.812875 25129 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:55:15.814451 25129 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 06:55:15.824779 25135 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:55:15.824779 25137 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:55:15.824774 25134 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:55:15.825086 25129 server_base.cc:1034] running on GCE node
I20260812 06:55:15.825541 25129 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:55:15.826191 25129 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:55:15.827366 25129 hybrid_clock.cc:648] HybridClock initialized: now 1786517715827356 us; error 42 us; skew 500 ppm
I20260812 06:55:15.829689 25129 webserver.cc:460] Webserver started at http://127.19.52.65:41977/ using document root <none> and password file <none>
I20260812 06:55:15.830330 25129 fs_manager.cc:359] Metadata directory not provided
I20260812 06:55:15.830400 25129 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:55:15.830802 25129 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 06:55:15.833359 25129 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786517687384214-19665-0/minicluster-data/ts-0/data/instance:
uuid: "f128f15afe164c94923d16797931caf8"
format_stamp: "Formatted at 2026-08-12 06:55:15 on dist-test-slave-65mx"
server_key: "40d9819466e82e7353c8bb207dd7d597"
server_key_iv: "81c193a7bf2be68a6d8d353e174d528a"
server_key_version: "encryptionkey@0"
I20260812 06:55:15.833978 25129 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786517687384214-19665-0/minicluster-data/ts-0/wal/instance:
uuid: "f128f15afe164c94923d16797931caf8"
format_stamp: "Formatted at 2026-08-12 06:55:15 on dist-test-slave-65mx"
server_key: "40d9819466e82e7353c8bb207dd7d597"
server_key_iv: "81c193a7bf2be68a6d8d353e174d528a"
server_key_version: "encryptionkey@0"
I20260812 06:55:15.837986 25129 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260812 06:55:15.840786 25145 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 06:55:15.842067 25129 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260812 06:55:15.842211 25129 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786517687384214-19665-0/minicluster-data/ts-0/data,/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786517687384214-19665-0/minicluster-data/ts-0/wal
uuid: "f128f15afe164c94923d16797931caf8"
format_stamp: "Formatted at 2026-08-12 06:55:15 on dist-test-slave-65mx"
server_key: "40d9819466e82e7353c8bb207dd7d597"
server_key_iv: "81c193a7bf2be68a6d8d353e174d528a"
server_key_version: "encryptionkey@0"
I20260812 06:55:15.842336 25129 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786517687384214-19665-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786517687384214-19665-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786517687384214-19665-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:55:15.846966 25129 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 06:55:15.847579 25129 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 06:55:15.847767 25129 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:55:15.848347 25129 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 06:55:15.849298 25129 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 06:55:15.849370 25129 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 06:55:15.849445 25129 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 06:55:15.849488 25129 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 06:55:15.859153 25129 rpc_server.cc:307] RPC server started. Bound to: 127.19.52.65:35651
I20260812 06:55:15.859185 25279 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.52.65:35651 every 8 connection(s)
I20260812 06:55:15.860203 25129 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786517687384214-19665-0/minicluster-data/ts-0/data/info.pb
I20260812 06:55:15.867527 25107 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:15.862295 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.52.65:54225 (local address 127.19.52.126:43689)
0812 06:55:15.862484 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0812 06:55:15.862489 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 06:55:15.863898 (+ 1409us) server_negotiation.cc:407] Connection header received
0812 06:55:15.865540 (+ 1642us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:55:15.865543 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:55:15.865590 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:55:15.865678 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:55:15.867039 (+ 1361us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:55:15.867042 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:55:15.867044 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 06:55:15.867059 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:55:15.867081 (+ 22us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 06:55:15.867089 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:55:15.867212 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0812 06:55:15.867339 (+ 127us) server_negotiation.cc:299] Negotiation successful
0812 06:55:15.867381 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260812 06:55:15.868491 25282 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:15.862781 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.126:43689 (local address 127.19.52.65:54225)
0812 06:55:15.863679 (+ 898us) negotiation.cc:107] Waiting for socket to connect
0812 06:55:15.863729 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0812 06:55:15.865236 (+ 1507us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:55:15.865873 (+ 637us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:55:15.865880 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:55:15.865979 (+ 99us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:55:15.866695 (+ 716us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 06:55:15.866696 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:55:15.866742 (+ 46us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 06:55:15.866818 (+ 76us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 06:55:15.867205 (+ 387us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 06:55:15.867212 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 06:55:15.867217 (+ 5us) client_negotiation.cc:769] Sending connection context
0812 06:55:15.867364 (+ 147us) client_negotiation.cc:240] Negotiation successful
0812 06:55:15.867582 (+ 218us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":353,"thread_start_us":134,"threads_started":1}
I20260812 06:55:15.869047 19665 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu as pid 25129
I20260812 06:55:15.869197 19665 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786517687384214-19665-0/minicluster-data/ts-0/wal/instance
I20260812 06:55:15.869554 19665 external_mini_cluster.cc:1428] Setting key 6af3abbe4cc2045979e2910a57fdffbd
I20260812 06:55:15.870069 25280 heartbeater.cc:347] Connected to a master server at 127.19.52.126:43689
I20260812 06:55:15.870347 25280 heartbeater.cc:464] Registering TS with master...
I20260812 06:55:15.870857 25280 heartbeater.cc:511] Master 127.19.52.126:43689 requested a full tablet report, sending...
I20260812 06:55:15.871767 19665 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu
/tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786517687384214-19665-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786517687384214-19665-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786517687384214-19665-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786517687384214-19665-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.52.66:0
--local_ip_for_outbound_sockets=127.19.52.66
--webserver_interface=127.19.52.66
--webserver_port=0
--tserver_master_addrs=127.19.52.126:43689
--builtin_ntp_servers=127.19.52.84:38249
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
I20260812 06:55:15.872645 25064 ts_manager.cc:194] Registered new tserver with Master: f128f15afe164c94923d16797931caf8 (127.19.52.65:35651)
W20260812 06:55:15.988778 25283 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:55:15.989099 25283 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:55:15.989174 25283 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 06:55:15.993419 25283 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 06:55:15.993520 25283 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 06:55:15.993614 25283 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.52.66
I20260812 06:55:15.998507 25283 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.52.84:38249
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786517687384214-19665-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786517687384214-19665-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.52.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786517687384214-19665-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.19.52.66
--webserver_port=0
--tserver_master_addrs=127.19.52.126:43689
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25283
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.52.66
--log_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786517687384214-19665-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 06:15:54 UTC on ccycloud.quasar-rnfzce.root.comops.site
build id 4581989
I20260812 06:55:15.999670 25283 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:55:16.001335 25283 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 06:55:16.011588 25292 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:55:16.011605 25290 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:55:16.011600 25289 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:55:16.011878 25283 server_base.cc:1034] running on GCE node
I20260812 06:55:16.012426 25283 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:55:16.013150 25283 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:55:16.014317 25283 hybrid_clock.cc:648] HybridClock initialized: now 1786517716014295 us; error 33 us; skew 500 ppm
I20260812 06:55:16.016501 25283 webserver.cc:460] Webserver started at http://127.19.52.66:45143/ using document root <none> and password file <none>
I20260812 06:55:16.017154 25283 fs_manager.cc:359] Metadata directory not provided
I20260812 06:55:16.017220 25283 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:55:16.017436 25283 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 06:55:16.019546 25283 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786517687384214-19665-0/minicluster-data/ts-1/data/instance:
uuid: "8d495be1f1c84a32b0eba6a96b95d499"
format_stamp: "Formatted at 2026-08-12 06:55:16 on dist-test-slave-65mx"
server_key: "a971c370ee0fa9176df38ae200214a31"
server_key_iv: "e8cb9c72a0eb87a1eb7fcb044f0d24d0"
server_key_version: "encryptionkey@0"
I20260812 06:55:16.020100 25283 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786517687384214-19665-0/minicluster-data/ts-1/wal/instance:
uuid: "8d495be1f1c84a32b0eba6a96b95d499"
format_stamp: "Formatted at 2026-08-12 06:55:16 on dist-test-slave-65mx"
server_key: "a971c370ee0fa9176df38ae200214a31"
server_key_iv: "e8cb9c72a0eb87a1eb7fcb044f0d24d0"
server_key_version: "encryptionkey@0"
I20260812 06:55:16.023896 25283 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.003s
I20260812 06:55:16.026717 25299 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 06:55:16.028106 25283 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260812 06:55:16.028262 25283 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786517687384214-19665-0/minicluster-data/ts-1/data,/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786517687384214-19665-0/minicluster-data/ts-1/wal
uuid: "8d495be1f1c84a32b0eba6a96b95d499"
format_stamp: "Formatted at 2026-08-12 06:55:16 on dist-test-slave-65mx"
server_key: "a971c370ee0fa9176df38ae200214a31"
server_key_iv: "e8cb9c72a0eb87a1eb7fcb044f0d24d0"
server_key_version: "encryptionkey@0"
I20260812 06:55:16.028399 25283 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786517687384214-19665-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786517687384214-19665-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786517687384214-19665-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:55:16.033411 25283 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 06:55:16.034085 25283 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 06:55:16.034279 25283 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:55:16.034860 25283 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 06:55:16.035827 25283 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 06:55:16.035877 25283 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 06:55:16.035940 25283 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 06:55:16.035975 25283 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 06:55:16.046177 25283 rpc_server.cc:307] RPC server started. Bound to: 127.19.52.66:38487
I20260812 06:55:16.046200 25431 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.52.66:38487 every 8 connection(s)
I20260812 06:55:16.047324 25283 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786517687384214-19665-0/minicluster-data/ts-1/data/info.pb
I20260812 06:55:16.049955 19665 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu as pid 25283
I20260812 06:55:16.050148 19665 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786517687384214-19665-0/minicluster-data/ts-1/wal/instance
I20260812 06:55:16.050500 19665 external_mini_cluster.cc:1428] Setting key 835be95ac425833d47d9a0c82a0b601b
I20260812 06:55:16.052889 19665 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu
/tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786517687384214-19665-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786517687384214-19665-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786517687384214-19665-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786517687384214-19665-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.52.67:0
--local_ip_for_outbound_sockets=127.19.52.67
--webserver_interface=127.19.52.67
--webserver_port=0
--tserver_master_addrs=127.19.52.126:43689
--builtin_ntp_servers=127.19.52.84:38249
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
I20260812 06:55:16.055855 25107 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:16.049347 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.52.66:44247 (local address 127.19.52.126:43689)
0812 06:55:16.049686 (+ 339us) server_negotiation.cc:206] Beginning negotiation
0812 06:55:16.049697 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0812 06:55:16.050815 (+ 1118us) server_negotiation.cc:407] Connection header received
0812 06:55:16.052820 (+ 2005us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:55:16.052824 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:55:16.052873 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:55:16.052933 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:55:16.055171 (+ 2238us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:55:16.055177 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:55:16.055182 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 06:55:16.055200 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:55:16.055230 (+ 30us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 06:55:16.055243 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:55:16.055366 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0812 06:55:16.055609 (+ 243us) server_negotiation.cc:299] Negotiation successful
0812 06:55:16.055671 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":229}
I20260812 06:55:16.056991 25434 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:16.049740 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.126:43689 (local address 127.19.52.66:44247)
0812 06:55:16.050635 (+ 895us) negotiation.cc:107] Waiting for socket to connect
0812 06:55:16.050701 (+ 66us) client_negotiation.cc:174] Beginning negotiation
0812 06:55:16.052489 (+ 1788us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:55:16.053388 (+ 899us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:55:16.053522 (+ 134us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:55:16.053763 (+ 241us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:55:16.054707 (+ 944us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 06:55:16.054711 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:55:16.054778 (+ 67us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 06:55:16.054893 (+ 115us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 06:55:16.055372 (+ 479us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 06:55:16.055381 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 06:55:16.055387 (+ 6us) client_negotiation.cc:769] Sending connection context
0812 06:55:16.055581 (+ 194us) client_negotiation.cc:240] Negotiation successful
0812 06:55:16.055826 (+ 245us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":357,"thread_start_us":451,"threads_started":1}
I20260812 06:55:16.058552 25432 heartbeater.cc:347] Connected to a master server at 127.19.52.126:43689
I20260812 06:55:16.058844 25432 heartbeater.cc:464] Registering TS with master...
I20260812 06:55:16.059351 25432 heartbeater.cc:511] Master 127.19.52.126:43689 requested a full tablet report, sending...
I20260812 06:55:16.061183 25064 ts_manager.cc:194] Registered new tserver with Master: 8d495be1f1c84a32b0eba6a96b95d499 (127.19.52.66:38487)
W20260812 06:55:16.180310 25435 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:55:16.180688 25435 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:55:16.180764 25435 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 06:55:16.185536 25435 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 06:55:16.185626 25435 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 06:55:16.185715 25435 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.52.67
I20260812 06:55:16.190917 25435 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.52.84:38249
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786517687384214-19665-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786517687384214-19665-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.52.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786517687384214-19665-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.19.52.67
--webserver_port=0
--tserver_master_addrs=127.19.52.126:43689
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25435
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.52.67
--log_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786517687384214-19665-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 06:15:54 UTC on ccycloud.quasar-rnfzce.root.comops.site
build id 4581989
I20260812 06:55:16.192095 25435 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:55:16.193588 25435 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 06:55:16.204103 25444 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:55:16.204126 25441 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:55:16.204155 25442 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:55:16.204346 25435 server_base.cc:1034] running on GCE node
I20260812 06:55:16.204957 25435 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:55:16.205654 25435 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:55:16.206825 25435 hybrid_clock.cc:648] HybridClock initialized: now 1786517716206797 us; error 45 us; skew 500 ppm
I20260812 06:55:16.208844 25435 webserver.cc:460] Webserver started at http://127.19.52.67:34575/ using document root <none> and password file <none>
I20260812 06:55:16.209388 25435 fs_manager.cc:359] Metadata directory not provided
I20260812 06:55:16.209451 25435 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:55:16.209630 25435 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 06:55:16.211697 25435 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786517687384214-19665-0/minicluster-data/ts-2/data/instance:
uuid: "e379a145a36440ffacf155ae0ed80a62"
format_stamp: "Formatted at 2026-08-12 06:55:16 on dist-test-slave-65mx"
server_key: "610a27a5f856ca713ee399e0f66fba49"
server_key_iv: "c27cbde1e9c923cfa23fd18ec186e893"
server_key_version: "encryptionkey@0"
I20260812 06:55:16.212225 25435 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786517687384214-19665-0/minicluster-data/ts-2/wal/instance:
uuid: "e379a145a36440ffacf155ae0ed80a62"
format_stamp: "Formatted at 2026-08-12 06:55:16 on dist-test-slave-65mx"
server_key: "610a27a5f856ca713ee399e0f66fba49"
server_key_iv: "c27cbde1e9c923cfa23fd18ec186e893"
server_key_version: "encryptionkey@0"
I20260812 06:55:16.216367 25435 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260812 06:55:16.219147 25453 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 06:55:16.220685 25435 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 06:55:16.220796 25435 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786517687384214-19665-0/minicluster-data/ts-2/data,/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786517687384214-19665-0/minicluster-data/ts-2/wal
uuid: "e379a145a36440ffacf155ae0ed80a62"
format_stamp: "Formatted at 2026-08-12 06:55:16 on dist-test-slave-65mx"
server_key: "610a27a5f856ca713ee399e0f66fba49"
server_key_iv: "c27cbde1e9c923cfa23fd18ec186e893"
server_key_version: "encryptionkey@0"
I20260812 06:55:16.220933 25435 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786517687384214-19665-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786517687384214-19665-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786517687384214-19665-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:55:16.225806 25435 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 06:55:16.226471 25435 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 06:55:16.226670 25435 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:55:16.227272 25435 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 06:55:16.228242 25435 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 06:55:16.228317 25435 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 06:55:16.228394 25435 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 06:55:16.228447 25435 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 06:55:16.238448 25435 rpc_server.cc:307] RPC server started. Bound to: 127.19.52.67:44339
I20260812 06:55:16.238556 25587 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.52.67:44339 every 8 connection(s)
I20260812 06:55:16.239506 25435 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786517687384214-19665-0/minicluster-data/ts-2/data/info.pb
I20260812 06:55:16.242353 19665 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu as pid 25435
I20260812 06:55:16.242506 19665 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786517687384214-19665-0/minicluster-data/ts-2/wal/instance
I20260812 06:55:16.242837 19665 external_mini_cluster.cc:1428] Setting key 4b200d8fd27ce05b14c9b3cadc459063
I20260812 06:55:16.246840 25107 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:16.241702 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.52.67:55843 (local address 127.19.52.126:43689)
0812 06:55:16.241870 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0812 06:55:16.241874 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 06:55:16.243256 (+ 1382us) server_negotiation.cc:407] Connection header received
0812 06:55:16.244924 (+ 1668us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:55:16.244928 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:55:16.244971 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:55:16.245048 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:55:16.246293 (+ 1245us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:55:16.246296 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:55:16.246299 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 06:55:16.246313 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:55:16.246337 (+ 24us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 06:55:16.246346 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:55:16.246469 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0812 06:55:16.246645 (+ 176us) server_negotiation.cc:299] Negotiation successful
0812 06:55:16.246695 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260812 06:55:16.247844 25592 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:16.242145 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.126:43689 (local address 127.19.52.67:55843)
0812 06:55:16.243029 (+ 884us) negotiation.cc:107] Waiting for socket to connect
0812 06:55:16.243089 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0812 06:55:16.244717 (+ 1628us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:55:16.245220 (+ 503us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:55:16.245229 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:55:16.245345 (+ 116us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:55:16.245994 (+ 649us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 06:55:16.245996 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:55:16.246059 (+ 63us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 06:55:16.246136 (+ 77us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 06:55:16.246469 (+ 333us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 06:55:16.246481 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 06:55:16.246490 (+ 9us) client_negotiation.cc:769] Sending connection context
0812 06:55:16.246681 (+ 191us) client_negotiation.cc:240] Negotiation successful
0812 06:55:16.247009 (+ 328us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":355,"thread_start_us":133,"threads_started":1}
I20260812 06:55:16.248998 25590 heartbeater.cc:347] Connected to a master server at 127.19.52.126:43689
I20260812 06:55:16.249259 25590 heartbeater.cc:464] Registering TS with master...
I20260812 06:55:16.249707 25590 heartbeater.cc:511] Master 127.19.52.126:43689 requested a full tablet report, sending...
I20260812 06:55:16.250787 25064 ts_manager.cc:194] Registered new tserver with Master: e379a145a36440ffacf155ae0ed80a62 (127.19.52.67:44339)
I20260812 06:55:16.257874 19665 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 06:55:16.262283 25107 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:16.260188 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36356 (local address 127.19.52.126:43689)
0812 06:55:16.260371 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0812 06:55:16.260382 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0812 06:55:16.260640 (+ 258us) server_negotiation.cc:407] Connection header received
0812 06:55:16.260816 (+ 176us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:55:16.260824 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:55:16.260867 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:55:16.260962 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:55:16.261842 (+ 880us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:55:16.261845 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:55:16.261848 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 06:55:16.261862 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:55:16.261891 (+ 29us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 06:55:16.261900 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:55:16.262004 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0812 06:55:16.262077 (+ 73us) server_negotiation.cc:299] Negotiation successful
0812 06:55:16.262128 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260812 06:55:16.266399 25064 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:36356:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 06:55:16.270596 25064 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 06:55:16.284417 25602 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:16.280627 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.66:38487 (local address 127.0.0.1:46504)
0812 06:55:16.281899 (+ 1272us) negotiation.cc:107] Waiting for socket to connect
0812 06:55:16.281962 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0812 06:55:16.282088 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:55:16.282601 (+ 513us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:55:16.282608 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:55:16.282687 (+ 79us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:55:16.283343 (+ 656us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 06:55:16.283345 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:55:16.283391 (+ 46us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 06:55:16.283464 (+ 73us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 06:55:16.283833 (+ 369us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 06:55:16.283842 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 06:55:16.283849 (+ 7us) client_negotiation.cc:769] Sending connection context
0812 06:55:16.284169 (+ 320us) client_negotiation.cc:240] Negotiation successful
0812 06:55:16.284248 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":661,"thread_start_us":112,"threads_started":1}
I20260812 06:55:16.284891 25604 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:16.280819 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46504 (local address 127.19.52.66:38487)
0812 06:55:16.281659 (+ 840us) server_negotiation.cc:206] Beginning negotiation
0812 06:55:16.281674 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0812 06:55:16.282099 (+ 425us) server_negotiation.cc:407] Connection header received
0812 06:55:16.282204 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:55:16.282210 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:55:16.282430 (+ 220us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:55:16.282601 (+ 171us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:55:16.283636 (+ 1035us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:55:16.283642 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:55:16.283655 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 06:55:16.283668 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:55:16.283715 (+ 47us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 06:55:16.283741 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:55:16.283861 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0812 06:55:16.284638 (+ 777us) server_negotiation.cc:299] Negotiation successful
0812 06:55:16.284724 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":734,"thread_start_us":104,"threads_started":1}
I20260812 06:55:16.285017 25607 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:16.281954 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49688 (local address 127.19.52.65:35651)
0812 06:55:16.282360 (+ 406us) server_negotiation.cc:206] Beginning negotiation
0812 06:55:16.282373 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0812 06:55:16.282398 (+ 25us) server_negotiation.cc:407] Connection header received
0812 06:55:16.282458 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:55:16.282464 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:55:16.282649 (+ 185us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:55:16.282813 (+ 164us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:55:16.284255 (+ 1442us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:55:16.284260 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:55:16.284270 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 06:55:16.284282 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:55:16.284322 (+ 40us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 06:55:16.284347 (+ 25us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:55:16.284455 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0812 06:55:16.284748 (+ 293us) server_negotiation.cc:299] Negotiation successful
0812 06:55:16.284826 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":293,"thread_start_us":140,"threads_started":1}
I20260812 06:55:16.284446 25603 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:16.280818 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.67:44339 (local address 127.0.0.1:35468)
0812 06:55:16.281899 (+ 1081us) negotiation.cc:107] Waiting for socket to connect
0812 06:55:16.281961 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0812 06:55:16.282073 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:55:16.282926 (+ 853us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:55:16.282930 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:55:16.282946 (+ 16us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:55:16.283502 (+ 556us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 06:55:16.283504 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:55:16.283515 (+ 11us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 06:55:16.283541 (+ 26us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 06:55:16.284138 (+ 597us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 06:55:16.284141 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 06:55:16.284145 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 06:55:16.284215 (+ 70us) client_negotiation.cc:240] Negotiation successful
0812 06:55:16.284258 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":924,"thread_start_us":107,"threads_started":1}
I20260812 06:55:16.285329 25605 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:16.281482 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.65:35651 (local address 127.0.0.1:49688)
0812 06:55:16.282300 (+ 818us) negotiation.cc:107] Waiting for socket to connect
0812 06:55:16.282312 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 06:55:16.282377 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:55:16.283030 (+ 653us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:55:16.283036 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:55:16.283053 (+ 17us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:55:16.283938 (+ 885us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 06:55:16.283942 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:55:16.283955 (+ 13us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 06:55:16.283978 (+ 23us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 06:55:16.284509 (+ 531us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 06:55:16.284512 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 06:55:16.284515 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 06:55:16.284641 (+ 126us) client_negotiation.cc:240] Negotiation successful
0812 06:55:16.284675 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":759,"thread_start_us":108,"threads_started":1}
I20260812 06:55:16.285740 25606 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:16.281522 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35468 (local address 127.19.52.67:44339)
0812 06:55:16.282485 (+ 963us) server_negotiation.cc:206] Beginning negotiation
0812 06:55:16.282497 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0812 06:55:16.282518 (+ 21us) server_negotiation.cc:407] Connection header received
0812 06:55:16.282566 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:55:16.282571 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:55:16.282738 (+ 167us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:55:16.283188 (+ 450us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:55:16.283906 (+ 718us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:55:16.283914 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:55:16.283925 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 06:55:16.283938 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:55:16.283982 (+ 44us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 06:55:16.284008 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:55:16.285403 (+ 1395us) server_negotiation.cc:1035] Waiting for connection context
0812 06:55:16.285469 (+ 66us) server_negotiation.cc:299] Negotiation successful
0812 06:55:16.285551 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":845,"thread_start_us":158,"threads_started":1}
I20260812 06:55:16.288233 25511 tablet_service.cc:1467] Processing CreateTablet for tablet 2e7df51b76704709b469fe918ce18961 (DEFAULT_TABLE table=test-table [id=fd5edda17e9a4810be5940094fbcf941]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 06:55:16.288233 25355 tablet_service.cc:1467] Processing CreateTablet for tablet 2e7df51b76704709b469fe918ce18961 (DEFAULT_TABLE table=test-table [id=fd5edda17e9a4810be5940094fbcf941]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 06:55:16.288273 25200 tablet_service.cc:1467] Processing CreateTablet for tablet 2e7df51b76704709b469fe918ce18961 (DEFAULT_TABLE table=test-table [id=fd5edda17e9a4810be5940094fbcf941]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 06:55:16.289397 25200 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2e7df51b76704709b469fe918ce18961. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 06:55:16.289402 25355 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2e7df51b76704709b469fe918ce18961. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 06:55:16.289402 25511 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2e7df51b76704709b469fe918ce18961. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 06:55:16.297333 25608 tablet_bootstrap.cc:492] T 2e7df51b76704709b469fe918ce18961 P f128f15afe164c94923d16797931caf8: Bootstrap starting.
I20260812 06:55:16.298065 25610 tablet_bootstrap.cc:492] T 2e7df51b76704709b469fe918ce18961 P e379a145a36440ffacf155ae0ed80a62: Bootstrap starting.
I20260812 06:55:16.298065 25609 tablet_bootstrap.cc:492] T 2e7df51b76704709b469fe918ce18961 P 8d495be1f1c84a32b0eba6a96b95d499: Bootstrap starting.
I20260812 06:55:16.299697 25608 tablet_bootstrap.cc:654] T 2e7df51b76704709b469fe918ce18961 P f128f15afe164c94923d16797931caf8: Neither blocks nor log segments found. Creating new log.
I20260812 06:55:16.300246 25610 tablet_bootstrap.cc:654] T 2e7df51b76704709b469fe918ce18961 P e379a145a36440ffacf155ae0ed80a62: Neither blocks nor log segments found. Creating new log.
I20260812 06:55:16.300246 25609 tablet_bootstrap.cc:654] T 2e7df51b76704709b469fe918ce18961 P 8d495be1f1c84a32b0eba6a96b95d499: Neither blocks nor log segments found. Creating new log.
I20260812 06:55:16.300714 25608 log.cc:824] T 2e7df51b76704709b469fe918ce18961 P f128f15afe164c94923d16797931caf8: Log is configured to *not* fsync() on all Append() calls
I20260812 06:55:16.301123 25610 log.cc:824] T 2e7df51b76704709b469fe918ce18961 P e379a145a36440ffacf155ae0ed80a62: Log is configured to *not* fsync() on all Append() calls
I20260812 06:55:16.301123 25609 log.cc:824] T 2e7df51b76704709b469fe918ce18961 P 8d495be1f1c84a32b0eba6a96b95d499: Log is configured to *not* fsync() on all Append() calls
I20260812 06:55:16.303231 25608 tablet_bootstrap.cc:492] T 2e7df51b76704709b469fe918ce18961 P f128f15afe164c94923d16797931caf8: No bootstrap required, opened a new log
I20260812 06:55:16.303229 25610 tablet_bootstrap.cc:492] T 2e7df51b76704709b469fe918ce18961 P e379a145a36440ffacf155ae0ed80a62: No bootstrap required, opened a new log
I20260812 06:55:16.303244 25609 tablet_bootstrap.cc:492] T 2e7df51b76704709b469fe918ce18961 P 8d495be1f1c84a32b0eba6a96b95d499: No bootstrap required, opened a new log
I20260812 06:55:16.303442 25608 ts_tablet_manager.cc:1397] T 2e7df51b76704709b469fe918ce18961 P f128f15afe164c94923d16797931caf8: Time spent bootstrapping tablet: real 0.007s user 0.000s sys 0.004s
I20260812 06:55:16.303442 25610 ts_tablet_manager.cc:1397] T 2e7df51b76704709b469fe918ce18961 P e379a145a36440ffacf155ae0ed80a62: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.004s
I20260812 06:55:16.303758 25609 ts_tablet_manager.cc:1397] T 2e7df51b76704709b469fe918ce18961 P 8d495be1f1c84a32b0eba6a96b95d499: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.005s
I20260812 06:55:16.307286 25609 raft_consensus.cc:348] T 2e7df51b76704709b469fe918ce18961 P 8d495be1f1c84a32b0eba6a96b95d499 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d495be1f1c84a32b0eba6a96b95d499" member_type: VOTER last_known_addr { host: "127.19.52.66" port: 38487 } } peers { permanent_uuid: "e379a145a36440ffacf155ae0ed80a62" member_type: VOTER last_known_addr { host: "127.19.52.67" port: 44339 } } peers { permanent_uuid: "f128f15afe164c94923d16797931caf8" member_type: VOTER last_known_addr { host: "127.19.52.65" port: 35651 } }
I20260812 06:55:16.307585 25609 raft_consensus.cc:374] T 2e7df51b76704709b469fe918ce18961 P 8d495be1f1c84a32b0eba6a96b95d499 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 06:55:16.307652 25609 raft_consensus.cc:729] T 2e7df51b76704709b469fe918ce18961 P 8d495be1f1c84a32b0eba6a96b95d499 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8d495be1f1c84a32b0eba6a96b95d499, State: Initialized, Role: FOLLOWER
I20260812 06:55:16.308125 25609 consensus_queue.cc:260] T 2e7df51b76704709b469fe918ce18961 P 8d495be1f1c84a32b0eba6a96b95d499 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d495be1f1c84a32b0eba6a96b95d499" member_type: VOTER last_known_addr { host: "127.19.52.66" port: 38487 } } peers { permanent_uuid: "e379a145a36440ffacf155ae0ed80a62" member_type: VOTER last_known_addr { host: "127.19.52.67" port: 44339 } } peers { permanent_uuid: "f128f15afe164c94923d16797931caf8" member_type: VOTER last_known_addr { host: "127.19.52.65" port: 35651 } }
I20260812 06:55:16.308238 25608 raft_consensus.cc:348] T 2e7df51b76704709b469fe918ce18961 P f128f15afe164c94923d16797931caf8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d495be1f1c84a32b0eba6a96b95d499" member_type: VOTER last_known_addr { host: "127.19.52.66" port: 38487 } } peers { permanent_uuid: "e379a145a36440ffacf155ae0ed80a62" member_type: VOTER last_known_addr { host: "127.19.52.67" port: 44339 } } peers { permanent_uuid: "f128f15afe164c94923d16797931caf8" member_type: VOTER last_known_addr { host: "127.19.52.65" port: 35651 } }
I20260812 06:55:16.308238 25610 raft_consensus.cc:348] T 2e7df51b76704709b469fe918ce18961 P e379a145a36440ffacf155ae0ed80a62 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d495be1f1c84a32b0eba6a96b95d499" member_type: VOTER last_known_addr { host: "127.19.52.66" port: 38487 } } peers { permanent_uuid: "e379a145a36440ffacf155ae0ed80a62" member_type: VOTER last_known_addr { host: "127.19.52.67" port: 44339 } } peers { permanent_uuid: "f128f15afe164c94923d16797931caf8" member_type: VOTER last_known_addr { host: "127.19.52.65" port: 35651 } }
I20260812 06:55:16.308517 25608 raft_consensus.cc:374] T 2e7df51b76704709b469fe918ce18961 P f128f15afe164c94923d16797931caf8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 06:55:16.308519 25610 raft_consensus.cc:374] T 2e7df51b76704709b469fe918ce18961 P e379a145a36440ffacf155ae0ed80a62 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 06:55:16.308609 25610 raft_consensus.cc:729] T 2e7df51b76704709b469fe918ce18961 P e379a145a36440ffacf155ae0ed80a62 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e379a145a36440ffacf155ae0ed80a62, State: Initialized, Role: FOLLOWER
I20260812 06:55:16.308619 25608 raft_consensus.cc:729] T 2e7df51b76704709b469fe918ce18961 P f128f15afe164c94923d16797931caf8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f128f15afe164c94923d16797931caf8, State: Initialized, Role: FOLLOWER
I20260812 06:55:16.309037 25432 heartbeater.cc:503] Master 127.19.52.126:43689 was elected leader, sending a full tablet report...
I20260812 06:55:16.309104 25610 consensus_queue.cc:260] T 2e7df51b76704709b469fe918ce18961 P e379a145a36440ffacf155ae0ed80a62 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d495be1f1c84a32b0eba6a96b95d499" member_type: VOTER last_known_addr { host: "127.19.52.66" port: 38487 } } peers { permanent_uuid: "e379a145a36440ffacf155ae0ed80a62" member_type: VOTER last_known_addr { host: "127.19.52.67" port: 44339 } } peers { permanent_uuid: "f128f15afe164c94923d16797931caf8" member_type: VOTER last_known_addr { host: "127.19.52.65" port: 35651 } }
I20260812 06:55:16.309104 25608 consensus_queue.cc:260] T 2e7df51b76704709b469fe918ce18961 P f128f15afe164c94923d16797931caf8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d495be1f1c84a32b0eba6a96b95d499" member_type: VOTER last_known_addr { host: "127.19.52.66" port: 38487 } } peers { permanent_uuid: "e379a145a36440ffacf155ae0ed80a62" member_type: VOTER last_known_addr { host: "127.19.52.67" port: 44339 } } peers { permanent_uuid: "f128f15afe164c94923d16797931caf8" member_type: VOTER last_known_addr { host: "127.19.52.65" port: 35651 } }
I20260812 06:55:16.310189 25610 ts_tablet_manager.cc:1428] T 2e7df51b76704709b469fe918ce18961 P e379a145a36440ffacf155ae0ed80a62: Time spent starting tablet: real 0.007s user 0.007s sys 0.000s
I20260812 06:55:16.310189 25608 ts_tablet_manager.cc:1428] T 2e7df51b76704709b469fe918ce18961 P f128f15afe164c94923d16797931caf8: Time spent starting tablet: real 0.007s user 0.008s sys 0.000s
I20260812 06:55:16.310552 25280 heartbeater.cc:503] Master 127.19.52.126:43689 was elected leader, sending a full tablet report...
I20260812 06:55:16.311131 25590 heartbeater.cc:503] Master 127.19.52.126:43689 was elected leader, sending a full tablet report...
I20260812 06:55:16.311522 25609 ts_tablet_manager.cc:1428] T 2e7df51b76704709b469fe918ce18961 P 8d495be1f1c84a32b0eba6a96b95d499: Time spent starting tablet: real 0.008s user 0.006s sys 0.000s
I20260812 06:55:16.347054 25617 raft_consensus.cc:482] T 2e7df51b76704709b469fe918ce18961 P f128f15afe164c94923d16797931caf8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 06:55:16.347301 25617 raft_consensus.cc:504] T 2e7df51b76704709b469fe918ce18961 P f128f15afe164c94923d16797931caf8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d495be1f1c84a32b0eba6a96b95d499" member_type: VOTER last_known_addr { host: "127.19.52.66" port: 38487 } } peers { permanent_uuid: "e379a145a36440ffacf155ae0ed80a62" member_type: VOTER last_known_addr { host: "127.19.52.67" port: 44339 } } peers { permanent_uuid: "f128f15afe164c94923d16797931caf8" member_type: VOTER last_known_addr { host: "127.19.52.65" port: 35651 } }
I20260812 06:55:16.348474 25617 leader_election.cc:288] T 2e7df51b76704709b469fe918ce18961 P f128f15afe164c94923d16797931caf8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8d495be1f1c84a32b0eba6a96b95d499 (127.19.52.66:38487), e379a145a36440ffacf155ae0ed80a62 (127.19.52.67:44339)
I20260812 06:55:16.350472 25282 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:16.348783 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.67:44339 (local address 127.19.52.65:51691)
0812 06:55:16.348982 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0812 06:55:16.348997 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 06:55:16.349099 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:55:16.349368 (+ 269us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:55:16.349370 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:55:16.349385 (+ 15us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:55:16.349880 (+ 495us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 06:55:16.349882 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:55:16.349896 (+ 14us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 06:55:16.349925 (+ 29us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 06:55:16.350225 (+ 300us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 06:55:16.350228 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 06:55:16.350231 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 06:55:16.350310 (+ 79us) client_negotiation.cc:240] Negotiation successful
0812 06:55:16.350345 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":111}
I20260812 06:55:16.350473 25606 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:16.348884 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.52.65:51691 (local address 127.19.52.67:44339)
0812 06:55:16.349027 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0812 06:55:16.349033 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 06:55:16.349052 (+ 19us) server_negotiation.cc:407] Connection header received
0812 06:55:16.349217 (+ 165us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:55:16.349220 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:55:16.349262 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:55:16.349351 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:55:16.350091 (+ 740us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:55:16.350095 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:55:16.350097 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 06:55:16.350109 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:55:16.350135 (+ 26us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 06:55:16.350142 (+ 7us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:55:16.350239 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0812 06:55:16.350305 (+ 66us) server_negotiation.cc:299] Negotiation successful
0812 06:55:16.350345 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260812 06:55:16.350706 25604 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:16.348938 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.52.65:54239 (local address 127.19.52.66:38487)
0812 06:55:16.349079 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0812 06:55:16.349084 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 06:55:16.349312 (+ 228us) server_negotiation.cc:407] Connection header received
0812 06:55:16.349480 (+ 168us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:55:16.349482 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:55:16.349516 (+ 34us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:55:16.349619 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:55:16.350290 (+ 671us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:55:16.350293 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:55:16.350296 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 06:55:16.350310 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:55:16.350333 (+ 23us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 06:55:16.350341 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:55:16.350437 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0812 06:55:16.350511 (+ 74us) server_negotiation.cc:299] Negotiation successful
0812 06:55:16.350554 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260812 06:55:16.351028 25618 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:16.348809 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.66:38487 (local address 127.19.52.65:54239)
0812 06:55:16.349212 (+ 403us) negotiation.cc:107] Waiting for socket to connect
0812 06:55:16.349224 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 06:55:16.349339 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:55:16.349660 (+ 321us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:55:16.349662 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:55:16.349673 (+ 11us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:55:16.350120 (+ 447us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 06:55:16.350122 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:55:16.350131 (+ 9us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 06:55:16.350156 (+ 25us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 06:55:16.350422 (+ 266us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 06:55:16.350425 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 06:55:16.350428 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 06:55:16.350514 (+ 86us) client_negotiation.cc:240] Negotiation successful
0812 06:55:16.350888 (+ 374us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":350,"thread_start_us":123,"threads_started":1}
I20260812 06:55:16.351110 25532 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2e7df51b76704709b469fe918ce18961" candidate_uuid: "f128f15afe164c94923d16797931caf8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e379a145a36440ffacf155ae0ed80a62" is_pre_election: true
I20260812 06:55:16.351440 25532 raft_consensus.cc:2454] T 2e7df51b76704709b469fe918ce18961 P e379a145a36440ffacf155ae0ed80a62 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f128f15afe164c94923d16797931caf8 in term 0.
I20260812 06:55:16.351624 25377 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2e7df51b76704709b469fe918ce18961" candidate_uuid: "f128f15afe164c94923d16797931caf8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8d495be1f1c84a32b0eba6a96b95d499" is_pre_election: true
I20260812 06:55:16.351931 25377 raft_consensus.cc:2454] T 2e7df51b76704709b469fe918ce18961 P 8d495be1f1c84a32b0eba6a96b95d499 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f128f15afe164c94923d16797931caf8 in term 0.
I20260812 06:55:16.352068 25146 leader_election.cc:302] T 2e7df51b76704709b469fe918ce18961 P f128f15afe164c94923d16797931caf8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e379a145a36440ffacf155ae0ed80a62, f128f15afe164c94923d16797931caf8; no voters:
I20260812 06:55:16.352392 25617 raft_consensus.cc:2781] T 2e7df51b76704709b469fe918ce18961 P f128f15afe164c94923d16797931caf8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 06:55:16.352510 25617 raft_consensus.cc:482] T 2e7df51b76704709b469fe918ce18961 P f128f15afe164c94923d16797931caf8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 06:55:16.352596 25617 raft_consensus.cc:3037] T 2e7df51b76704709b469fe918ce18961 P f128f15afe164c94923d16797931caf8 [term 0 FOLLOWER]: Advancing to term 1
I20260812 06:55:16.353664 25617 raft_consensus.cc:504] T 2e7df51b76704709b469fe918ce18961 P f128f15afe164c94923d16797931caf8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d495be1f1c84a32b0eba6a96b95d499" member_type: VOTER last_known_addr { host: "127.19.52.66" port: 38487 } } peers { permanent_uuid: "e379a145a36440ffacf155ae0ed80a62" member_type: VOTER last_known_addr { host: "127.19.52.67" port: 44339 } } peers { permanent_uuid: "f128f15afe164c94923d16797931caf8" member_type: VOTER last_known_addr { host: "127.19.52.65" port: 35651 } }
I20260812 06:55:16.354039 25617 leader_election.cc:288] T 2e7df51b76704709b469fe918ce18961 P f128f15afe164c94923d16797931caf8 [CANDIDATE]: Term 1 election: Requested vote from peers 8d495be1f1c84a32b0eba6a96b95d499 (127.19.52.66:38487), e379a145a36440ffacf155ae0ed80a62 (127.19.52.67:44339)
I20260812 06:55:16.354436 25377 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2e7df51b76704709b469fe918ce18961" candidate_uuid: "f128f15afe164c94923d16797931caf8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8d495be1f1c84a32b0eba6a96b95d499"
I20260812 06:55:16.354606 25377 raft_consensus.cc:3037] T 2e7df51b76704709b469fe918ce18961 P 8d495be1f1c84a32b0eba6a96b95d499 [term 0 FOLLOWER]: Advancing to term 1
I20260812 06:55:16.354609 25532 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2e7df51b76704709b469fe918ce18961" candidate_uuid: "f128f15afe164c94923d16797931caf8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e379a145a36440ffacf155ae0ed80a62"
I20260812 06:55:16.354790 25532 raft_consensus.cc:3037] T 2e7df51b76704709b469fe918ce18961 P e379a145a36440ffacf155ae0ed80a62 [term 0 FOLLOWER]: Advancing to term 1
I20260812 06:55:16.356050 25532 raft_consensus.cc:2454] T 2e7df51b76704709b469fe918ce18961 P e379a145a36440ffacf155ae0ed80a62 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f128f15afe164c94923d16797931caf8 in term 1.
I20260812 06:55:16.356448 25377 raft_consensus.cc:2454] T 2e7df51b76704709b469fe918ce18961 P 8d495be1f1c84a32b0eba6a96b95d499 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f128f15afe164c94923d16797931caf8 in term 1.
I20260812 06:55:16.356514 25146 leader_election.cc:302] T 2e7df51b76704709b469fe918ce18961 P f128f15afe164c94923d16797931caf8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e379a145a36440ffacf155ae0ed80a62, f128f15afe164c94923d16797931caf8; no voters:
I20260812 06:55:16.356930 25617 raft_consensus.cc:2781] T 2e7df51b76704709b469fe918ce18961 P f128f15afe164c94923d16797931caf8 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 06:55:16.357209 25617 raft_consensus.cc:686] T 2e7df51b76704709b469fe918ce18961 P f128f15afe164c94923d16797931caf8 [term 1 LEADER]: Becoming Leader. State: Replica: f128f15afe164c94923d16797931caf8, State: Running, Role: LEADER
I20260812 06:55:16.357720 25617 consensus_queue.cc:237] T 2e7df51b76704709b469fe918ce18961 P f128f15afe164c94923d16797931caf8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d495be1f1c84a32b0eba6a96b95d499" member_type: VOTER last_known_addr { host: "127.19.52.66" port: 38487 } } peers { permanent_uuid: "e379a145a36440ffacf155ae0ed80a62" member_type: VOTER last_known_addr { host: "127.19.52.67" port: 44339 } } peers { permanent_uuid: "f128f15afe164c94923d16797931caf8" member_type: VOTER last_known_addr { host: "127.19.52.65" port: 35651 } }
W20260812 06:55:16.361495 25281 tablet.cc:2369] T 2e7df51b76704709b469fe918ce18961 P f128f15afe164c94923d16797931caf8: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 06:55:16.362262 25063 catalog_manager.cc:5495] T 2e7df51b76704709b469fe918ce18961 P f128f15afe164c94923d16797931caf8 reported cstate change: term changed from 0 to 1, leader changed from <none> to f128f15afe164c94923d16797931caf8 (127.19.52.65). New cstate: current_term: 1 leader_uuid: "f128f15afe164c94923d16797931caf8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d495be1f1c84a32b0eba6a96b95d499" member_type: VOTER last_known_addr { host: "127.19.52.66" port: 38487 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e379a145a36440ffacf155ae0ed80a62" member_type: VOTER last_known_addr { host: "127.19.52.67" port: 44339 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f128f15afe164c94923d16797931caf8" member_type: VOTER last_known_addr { host: "127.19.52.65" port: 35651 } health_report { overall_health: HEALTHY } } }
I20260812 06:55:16.386956 25607 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:16.385390 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49700 (local address 127.19.52.65:35651)
0812 06:55:16.385564 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0812 06:55:16.385568 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 06:55:16.385583 (+ 15us) server_negotiation.cc:407] Connection header received
0812 06:55:16.385667 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:55:16.385670 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:55:16.385733 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:55:16.385823 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:55:16.386577 (+ 754us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:55:16.386580 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:55:16.386582 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 06:55:16.386596 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:55:16.386616 (+ 20us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 06:55:16.386625 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:55:16.386718 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0812 06:55:16.386797 (+ 79us) server_negotiation.cc:299] Negotiation successful
0812 06:55:16.386835 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260812 06:55:16.393374 25377 raft_consensus.cc:1261] T 2e7df51b76704709b469fe918ce18961 P 8d495be1f1c84a32b0eba6a96b95d499 [term 1 FOLLOWER]: Refusing update from remote peer f128f15afe164c94923d16797931caf8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 06:55:16.393374 25532 raft_consensus.cc:1261] T 2e7df51b76704709b469fe918ce18961 P e379a145a36440ffacf155ae0ed80a62 [term 1 FOLLOWER]: Refusing update from remote peer f128f15afe164c94923d16797931caf8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 06:55:16.394191 25617 consensus_queue.cc:1035] T 2e7df51b76704709b469fe918ce18961 P f128f15afe164c94923d16797931caf8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e379a145a36440ffacf155ae0ed80a62" member_type: VOTER last_known_addr { host: "127.19.52.67" port: 44339 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 06:55:16.394424 25619 consensus_queue.cc:1035] T 2e7df51b76704709b469fe918ce18961 P f128f15afe164c94923d16797931caf8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8d495be1f1c84a32b0eba6a96b95d499" member_type: VOTER last_known_addr { host: "127.19.52.66" port: 38487 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 06:55:16.402089 25626 mvcc.cc:205] Tried to move back new op lower bound from 7317576566339260416 to 7317576566205116416. Current Snapshot: MvccSnapshot[applied={T|T < 7317576566339260416}]
I20260812 06:55:16.412748 25063 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:36356:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 06:55:16.412968 25063 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:36356:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 06:55:16.415840 25063 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b7e4ad509c3f4c19a716829db7c9ed79: Sending DeleteTablet for 3 replicas of tablet 2e7df51b76704709b469fe918ce18961
I20260812 06:55:16.416868 25355 tablet_service.cc:1514] Processing DeleteTablet for tablet 2e7df51b76704709b469fe918ce18961 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 06:55:16 UTC) from {username='slave'} at 127.0.0.1:46504
I20260812 06:55:16.417039 25200 tablet_service.cc:1514] Processing DeleteTablet for tablet 2e7df51b76704709b469fe918ce18961 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 06:55:16 UTC) from {username='slave'} at 127.0.0.1:49688
I20260812 06:55:16.417549 25635 tablet_replica.cc:330] T 2e7df51b76704709b469fe918ce18961 P f128f15afe164c94923d16797931caf8: stopping tablet replica
I20260812 06:55:16.417485 25511 tablet_service.cc:1514] Processing DeleteTablet for tablet 2e7df51b76704709b469fe918ce18961 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 06:55:16 UTC) from {username='slave'} at 127.0.0.1:35468
I20260812 06:55:16.417912 25635 raft_consensus.cc:2229] T 2e7df51b76704709b469fe918ce18961 P f128f15afe164c94923d16797931caf8 [term 1 LEADER]: Raft consensus shutting down.
I20260812 06:55:16.418056 25636 tablet_replica.cc:330] T 2e7df51b76704709b469fe918ce18961 P e379a145a36440ffacf155ae0ed80a62: stopping tablet replica
I20260812 06:55:16.418085 19665 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu with pid 25129
I20260812 06:55:16.418412 25635 raft_consensus.cc:2258] T 2e7df51b76704709b469fe918ce18961 P f128f15afe164c94923d16797931caf8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 06:55:16.418426 25636 raft_consensus.cc:2229] T 2e7df51b76704709b469fe918ce18961 P e379a145a36440ffacf155ae0ed80a62 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 06:55:16.418933 25636 raft_consensus.cc:2258] T 2e7df51b76704709b469fe918ce18961 P e379a145a36440ffacf155ae0ed80a62 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 06:55:16.420943 25636 ts_tablet_manager.cc:1905] T 2e7df51b76704709b469fe918ce18961 P e379a145a36440ffacf155ae0ed80a62: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 06:55:16.424427 25636 ts_tablet_manager.cc:1918] T 2e7df51b76704709b469fe918ce18961 P e379a145a36440ffacf155ae0ed80a62: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 06:55:16.424661 25636 log.cc:1196] T 2e7df51b76704709b469fe918ce18961 P e379a145a36440ffacf155ae0ed80a62: Deleting WAL directory at /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786517687384214-19665-0/minicluster-data/ts-2/wal/wals/2e7df51b76704709b469fe918ce18961
I20260812 06:55:16.425019 25633 tablet_replica.cc:330] T 2e7df51b76704709b469fe918ce18961 P 8d495be1f1c84a32b0eba6a96b95d499: stopping tablet replica
I20260812 06:55:16.425036 25636 ts_tablet_manager.cc:1939] T 2e7df51b76704709b469fe918ce18961 P e379a145a36440ffacf155ae0ed80a62: Deleting consensus metadata
I20260812 06:55:16.425487 25633 raft_consensus.cc:2229] T 2e7df51b76704709b469fe918ce18961 P 8d495be1f1c84a32b0eba6a96b95d499 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 06:55:16.425899 25633 raft_consensus.cc:2258] T 2e7df51b76704709b469fe918ce18961 P 8d495be1f1c84a32b0eba6a96b95d499 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 06:55:16.426652 25042 catalog_manager.cc:4841] TS e379a145a36440ffacf155ae0ed80a62 (127.19.52.67:44339): tablet 2e7df51b76704709b469fe918ce18961 (table test-table [id=fd5edda17e9a4810be5940094fbcf941]) successfully deleted
I20260812 06:55:16.427680 25633 ts_tablet_manager.cc:1905] T 2e7df51b76704709b469fe918ce18961 P 8d495be1f1c84a32b0eba6a96b95d499: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 06:55:16.430270 25633 ts_tablet_manager.cc:1918] T 2e7df51b76704709b469fe918ce18961 P 8d495be1f1c84a32b0eba6a96b95d499: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 06:55:16.430408 25633 log.cc:1196] T 2e7df51b76704709b469fe918ce18961 P 8d495be1f1c84a32b0eba6a96b95d499: Deleting WAL directory at /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786517687384214-19665-0/minicluster-data/ts-1/wal/wals/2e7df51b76704709b469fe918ce18961
I20260812 06:55:16.430748 25633 ts_tablet_manager.cc:1939] T 2e7df51b76704709b469fe918ce18961 P 8d495be1f1c84a32b0eba6a96b95d499: Deleting consensus metadata
W20260812 06:55:16.431036 25043 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.19.52.65:35651 (error 108)
I20260812 06:55:16.431290 19665 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu with pid 25283
I20260812 06:55:16.432529 25605 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:16.432035 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.65:35651 (local address 127.0.0.1:49702)
0812 06:55:16.432300 (+ 265us) negotiation.cc:107] Waiting for socket to connect
0812 06:55:16.432395 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.52.65:35651: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":131}
W20260812 06:55:16.432744 25043 catalog_manager.cc:4584] TS f128f15afe164c94923d16797931caf8 (127.19.52.65:35651): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 2e7df51b76704709b469fe918ce18961: Network error: Client connection negotiation failed: client connection to 127.19.52.65:35651: connect: Connection refused (error 111)
I20260812 06:55:16.438793 19665 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu with pid 25435
I20260812 06:55:16.442304 25605 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:16.439030 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.66:38487 (local address 127.0.0.1:46508)
0812 06:55:16.442097 (+ 3067us) negotiation.cc:107] Waiting for socket to connect
0812 06:55:16.442177 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.52.66:38487: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":2931}
I20260812 06:55:16.447304 19665 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu with pid 25021
2026-08-12T06:55:16Z chronyd exiting
[ OK ] SecurityITest.TestDisableAuthenticationEncryption (1074 ms)
[ RUN ] SecurityITest.TestJwtMiniCluster
I20260812 06:55:16.475307 19665 mini_oidc.cc:152] Starting JWKS server
I20260812 06:55:16.477326 19665 webserver.cc:460] Webserver started at https://127.0.0.1:34753/ using document root <none> and password file <none>
I20260812 06:55:16.477535 19665 mini_oidc.cc:189] Starting OIDC Discovery server
I20260812 06:55:16.477867 19665 webserver.cc:460] Webserver started at http://127.0.0.1:41741/ using document root <none> and password file <none>
2026-08-12T06:55:16Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T06:55:16Z Disabled control of system clock
I20260812 06:55:16.490566 19665 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu
/tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786517687384214-19665-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786517687384214-19665-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786517687384214-19665-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786517687384214-19665-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.52.126:45009
--webserver_interface=127.19.52.126
--webserver_port=0
--builtin_ntp_servers=127.19.52.84:42409
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.19.52.126:45009
--enable_jwt_token_auth=true
--jwks_url=https://localhost:34753/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786517687384214-19665-0/testchainca.cert with env {}
W20260812 06:55:16.615257 25645 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:55:16.615607 25645 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:55:16.615653 25645 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 06:55:16.620143 25645 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 06:55:16.620229 25645 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 06:55:16.620245 25645 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 06:55:16.620262 25645 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 06:55:16.620278 25645 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260812 06:55:16.620296 25645 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260812 06:55:16.620308 25645 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:34753/jwks.json
I20260812 06:55:16.626706 25645 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.52.84:42409
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786517687384214-19665-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786517687384214-19665-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.19.52.126:45009
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.52.126:45009
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:34753/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786517687384214-19665-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.19.52.126
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786517687384214-19665-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25645
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786517687384214-19665-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 06:15:54 UTC on ccycloud.quasar-rnfzce.root.comops.site
build id 4581989
I20260812 06:55:16.627976 25645 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:55:16.629060 25645 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 06:55:16.638332 25651 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:55:16.638370 25654 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:55:16.638332 25652 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:55:16.638960 25645 server_base.cc:1034] running on GCE node
I20260812 06:55:16.639614 25645 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:55:16.640970 25645 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:55:16.642161 25645 hybrid_clock.cc:648] HybridClock initialized: now 1786517716642126 us; error 50 us; skew 500 ppm
I20260812 06:55:16.644338 25645 webserver.cc:460] Webserver started at http://127.19.52.126:39441/ using document root <none> and password file <none>
I20260812 06:55:16.645027 25645 fs_manager.cc:359] Metadata directory not provided
I20260812 06:55:16.645108 25645 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:55:16.645339 25645 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 06:55:16.647511 25645 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786517687384214-19665-0/minicluster-data/master-0/data/instance:
uuid: "f7c87f509b46436187ef5809df1723f9"
format_stamp: "Formatted at 2026-08-12 06:55:16 on dist-test-slave-65mx"
server_key: "3810d5c13457c5edeb37b80c69381ca9"
server_key_iv: "c7f96bc13583a16d3dfedadd197430b0"
server_key_version: "encryptionkey@0"
I20260812 06:55:16.648136 25645 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786517687384214-19665-0/minicluster-data/master-0/wal/instance:
uuid: "f7c87f509b46436187ef5809df1723f9"
format_stamp: "Formatted at 2026-08-12 06:55:16 on dist-test-slave-65mx"
server_key: "3810d5c13457c5edeb37b80c69381ca9"
server_key_iv: "c7f96bc13583a16d3dfedadd197430b0"
server_key_version: "encryptionkey@0"
I20260812 06:55:16.652837 25645 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.007s sys 0.000s
I20260812 06:55:16.656098 25660 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 06:55:16.658038 25645 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 06:55:16.658237 25645 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786517687384214-19665-0/minicluster-data/master-0/data,/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786517687384214-19665-0/minicluster-data/master-0/wal
uuid: "f7c87f509b46436187ef5809df1723f9"
format_stamp: "Formatted at 2026-08-12 06:55:16 on dist-test-slave-65mx"
server_key: "3810d5c13457c5edeb37b80c69381ca9"
server_key_iv: "c7f96bc13583a16d3dfedadd197430b0"
server_key_version: "encryptionkey@0"
I20260812 06:55:16.658408 25645 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786517687384214-19665-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786517687384214-19665-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786517687384214-19665-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:55:16.676832 25645 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 06:55:16.677729 25645 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 06:55:16.677951 25645 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:55:16.686215 25645 rpc_server.cc:307] RPC server started. Bound to: 127.19.52.126:45009
I20260812 06:55:16.686252 25721 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.52.126:45009 every 8 connection(s)
I20260812 06:55:16.687373 25645 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786517687384214-19665-0/minicluster-data/master-0/data/info.pb
I20260812 06:55:16.688261 19665 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu as pid 25645
I20260812 06:55:16.688401 19665 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786517687384214-19665-0/minicluster-data/master-0/wal/instance
I20260812 06:55:16.688781 19665 external_mini_cluster.cc:1428] Setting key 123affeb1e7defc7c11d922643123683
I20260812 06:55:16.691830 25723 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 06:55:16.699393 25723 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f7c87f509b46436187ef5809df1723f9: Bootstrap starting.
I20260812 06:55:16.700472 25726 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:16.690248 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43432 (local address 127.19.52.126:45009)
0812 06:55:16.691029 (+ 781us) server_negotiation.cc:206] Beginning negotiation
0812 06:55:16.691067 (+ 38us) server_negotiation.cc:399] Waiting for connection header
0812 06:55:16.691115 (+ 48us) server_negotiation.cc:407] Connection header received
0812 06:55:16.692048 (+ 933us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:55:16.692072 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:55:16.692633 (+ 561us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:55:16.693017 (+ 384us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:55:16.693502 (+ 485us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:16.695022 (+ 1520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:16.696126 (+ 1104us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:16.696897 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:16.697097 (+ 200us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:55:16.698063 (+ 966us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:55:16.698069 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:55:16.698088 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 06:55:16.698656 (+ 568us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:55:16.698735 (+ 79us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 06:55:16.698762 (+ 27us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:55:16.698872 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0812 06:55:16.699126 (+ 254us) server_negotiation.cc:299] Negotiation successful
0812 06:55:16.699423 (+ 297us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":354,"thread_start_us":143,"threads_started":1}
I20260812 06:55:16.702287 25723 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f7c87f509b46436187ef5809df1723f9: Neither blocks nor log segments found. Creating new log.
I20260812 06:55:16.703258 25723 log.cc:824] T 00000000000000000000000000000000 P f7c87f509b46436187ef5809df1723f9: Log is configured to *not* fsync() on all Append() calls
I20260812 06:55:16.704324 19665 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 06:55:16.705890 25723 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f7c87f509b46436187ef5809df1723f9: No bootstrap required, opened a new log
I20260812 06:55:16.709648 25723 raft_consensus.cc:348] T 00000000000000000000000000000000 P f7c87f509b46436187ef5809df1723f9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f7c87f509b46436187ef5809df1723f9" member_type: VOTER last_known_addr { host: "127.19.52.126" port: 45009 } }
I20260812 06:55:16.709923 25723 raft_consensus.cc:374] T 00000000000000000000000000000000 P f7c87f509b46436187ef5809df1723f9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 06:55:16.709960 25723 raft_consensus.cc:729] T 00000000000000000000000000000000 P f7c87f509b46436187ef5809df1723f9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f7c87f509b46436187ef5809df1723f9, State: Initialized, Role: FOLLOWER
I20260812 06:55:16.710430 25723 consensus_queue.cc:260] T 00000000000000000000000000000000 P f7c87f509b46436187ef5809df1723f9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f7c87f509b46436187ef5809df1723f9" member_type: VOTER last_known_addr { host: "127.19.52.126" port: 45009 } }
I20260812 06:55:16.710716 25723 raft_consensus.cc:388] T 00000000000000000000000000000000 P f7c87f509b46436187ef5809df1723f9 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 06:55:16.710826 25723 raft_consensus.cc:482] T 00000000000000000000000000000000 P f7c87f509b46436187ef5809df1723f9 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 06:55:16.710952 25723 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f7c87f509b46436187ef5809df1723f9 [term 0 FOLLOWER]: Advancing to term 1
I20260812 06:55:16.712119 25723 raft_consensus.cc:504] T 00000000000000000000000000000000 P f7c87f509b46436187ef5809df1723f9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f7c87f509b46436187ef5809df1723f9" member_type: VOTER last_known_addr { host: "127.19.52.126" port: 45009 } }
I20260812 06:55:16.712517 25723 leader_election.cc:302] T 00000000000000000000000000000000 P f7c87f509b46436187ef5809df1723f9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f7c87f509b46436187ef5809df1723f9; no voters:
I20260812 06:55:16.712949 25723 leader_election.cc:288] T 00000000000000000000000000000000 P f7c87f509b46436187ef5809df1723f9 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 06:55:16.713158 25728 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f7c87f509b46436187ef5809df1723f9 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 06:55:16.713424 25728 raft_consensus.cc:686] T 00000000000000000000000000000000 P f7c87f509b46436187ef5809df1723f9 [term 1 LEADER]: Becoming Leader. State: Replica: f7c87f509b46436187ef5809df1723f9, State: Running, Role: LEADER
I20260812 06:55:16.713788 25728 consensus_queue.cc:237] T 00000000000000000000000000000000 P f7c87f509b46436187ef5809df1723f9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f7c87f509b46436187ef5809df1723f9" member_type: VOTER last_known_addr { host: "127.19.52.126" port: 45009 } }
I20260812 06:55:16.714398 25723 sys_catalog.cc:564] T 00000000000000000000000000000000 P f7c87f509b46436187ef5809df1723f9 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 06:55:16.715690 25730 sys_catalog.cc:455] T 00000000000000000000000000000000 P f7c87f509b46436187ef5809df1723f9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f7c87f509b46436187ef5809df1723f9. Latest consensus state: current_term: 1 leader_uuid: "f7c87f509b46436187ef5809df1723f9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f7c87f509b46436187ef5809df1723f9" member_type: VOTER last_known_addr { host: "127.19.52.126" port: 45009 } } }
I20260812 06:55:16.715832 25730 sys_catalog.cc:458] T 00000000000000000000000000000000 P f7c87f509b46436187ef5809df1723f9 [sys.catalog]: This master's current role is: LEADER
I20260812 06:55:16.715667 25729 sys_catalog.cc:455] T 00000000000000000000000000000000 P f7c87f509b46436187ef5809df1723f9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f7c87f509b46436187ef5809df1723f9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f7c87f509b46436187ef5809df1723f9" member_type: VOTER last_known_addr { host: "127.19.52.126" port: 45009 } } }
I20260812 06:55:16.716032 25729 sys_catalog.cc:458] T 00000000000000000000000000000000 P f7c87f509b46436187ef5809df1723f9 [sys.catalog]: This master's current role is: LEADER
I20260812 06:55:16.716419 25739 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 06:55:16.720419 25739 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 06:55:16.727855 25739 catalog_manager.cc:1347] Generated new cluster ID: 2dbda5bf85134dd58eff2b6223d4b738
I20260812 06:55:16.727962 25739 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 06:55:16.739996 25739 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 06:55:16.741283 25739 catalog_manager.cc:1504] Loading token signing keys...
I20260812 06:55:16.747071 25739 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f7c87f509b46436187ef5809df1723f9: Generated new TSK 0
I20260812 06:55:16.747862 25739 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260812 06:55:16.786410 25757 webserver.cc:533] Webserver: sslize failed: error:14094415:SSL routines:ssl3_read_bytes:sslv3 alert certificate expired
E20260812 06:55:16.786599 25757 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260812 06:55:16.787189 25726 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:16.761500 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43442 (local address 127.19.52.126:45009)
0812 06:55:16.761653 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0812 06:55:16.761663 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0812 06:55:16.762015 (+ 352us) server_negotiation.cc:407] Connection header received
0812 06:55:16.762310 (+ 295us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:55:16.762313 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:55:16.762369 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:55:16.762483 (+ 114us) server_negotiation.cc:226] Negotiated authn=JWT
0812 06:55:16.762809 (+ 326us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:16.763424 (+ 615us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:16.764357 (+ 933us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:16.764667 (+ 310us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:16.764778 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:55:16.765216 (+ 438us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0812 06:55:16.786633 (+ 21417us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:34753/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0812 06:55:16.786970 (+ 337us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:43442: Failed to load JWKS: Error downloading JWKS from 'https://localhost:34753/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"server-negotiator.queue_time_us":52}
W20260812 06:55:16.787281 25726 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:43442: Failed to load JWKS: Error downloading JWKS from 'https://localhost:34753/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
W20260812 06:55:16.788607 25756 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 06:55:16.761328 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.126:45009 (local address 127.0.0.1:43442)
0812 06:55:16.761878 (+ 550us) negotiation.cc:107] Waiting for socket to connect
0812 06:55:16.761892 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 06:55:16.762112 (+ 220us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:55:16.762515 (+ 403us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:55:16.762519 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:55:16.762527 (+ 8us) client_negotiation.cc:189] Negotiated authn=JWT
0812 06:55:16.762682 (+ 155us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:55:16.762688 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:55:16.763561 (+ 873us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:55:16.763565 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:55:16.764207 (+ 642us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:55:16.764215 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:55:16.764804 (+ 589us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:55:16.764818 (+ 14us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:55:16.764922 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:55:16.764983 (+ 61us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0812 06:55:16.786914 (+ 21931us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:34753/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0812 06:55:16.787181 (+ 267us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.19.52.126:45009: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:34753/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"client-negotiator.queue_time_us":359,"thread_start_us":224,"threads_started":1}
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:597: Failure
Failed
Bad status: Runtime error: Could not connect to the cluster: Client connection negotiation failed: client connection to 127.19.52.126:45009: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:34753/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Google Test trace:
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:593: Valid JWT
I20260812 06:55:16.789390 19665 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu with pid 25645
2026-08-12T06:55:16Z chronyd exiting
I20260812 06:55:16.829627 19665 test_util.cc:175] -----------------------------------------------
I20260812 06:55:16.829744 19665 test_util.cc:176] Had fatal failures, leaving test files at /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786517687384214-19665-0
[ FAILED ] SecurityITest.TestJwtMiniCluster (356 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithInvalidCert
I20260812 06:55:16.830983 19665 mini_oidc.cc:152] Starting JWKS server
I20260812 06:55:16.831951 19665 webserver.cc:460] Webserver started at https://127.0.0.1:44423/ using document root <none> and password file <none>
I20260812 06:55:16.832099 19665 mini_oidc.cc:189] Starting OIDC Discovery server
I20260812 06:55:16.832417 19665 webserver.cc:460] Webserver started at http://127.0.0.1:44673/ using document root <none> and password file <none>
2026-08-12T06:55:16Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T06:55:16Z Disabled control of system clock
I20260812 06:55:16.843189 19665 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu
/tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786517687384214-19665-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786517687384214-19665-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786517687384214-19665-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786517687384214-19665-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.52.126:40473
--webserver_interface=127.19.52.126
--webserver_port=0
--builtin_ntp_servers=127.19.52.84:45895
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.19.52.126:40473
--enable_jwt_token_auth=true
--jwks_url=https://localhost:44423/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786517687384214-19665-0/testchainca.cert with env {}
W20260812 06:55:16.963253 25764 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:55:16.963639 25764 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:55:16.963722 25764 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 06:55:16.968184 25764 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 06:55:16.968327 25764 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 06:55:16.968355 25764 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 06:55:16.968387 25764 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 06:55:16.968423 25764 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260812 06:55:16.968461 25764 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260812 06:55:16.968490 25764 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:44423/jwks.json
I20260812 06:55:16.974741 25764 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.52.84:45895
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786517687384214-19665-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786517687384214-19665-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.19.52.126:40473
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.52.126:40473
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:44423/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786517687384214-19665-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.19.52.126
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786517687384214-19665-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25764
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786517687384214-19665-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 06:15:54 UTC on ccycloud.quasar-rnfzce.root.comops.site
build id 4581989
I20260812 06:55:16.976024 25764 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:55:16.977200 25764 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 06:55:16.986182 25773 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:55:16.986222 25770 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:55:16.986274 25764 server_base.cc:1034] running on GCE node
W20260812 06:55:16.986209 25769 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:55:16.987018 25764 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:55:16.988106 25764 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:55:16.989279 25764 hybrid_clock.cc:648] HybridClock initialized: now 1786517716989247 us; error 43 us; skew 500 ppm
I20260812 06:55:16.991236 25764 webserver.cc:460] Webserver started at http://127.19.52.126:37083/ using document root <none> and password file <none>
I20260812 06:55:16.991835 25764 fs_manager.cc:359] Metadata directory not provided
I20260812 06:55:16.991899 25764 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:55:16.992132 25764 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 06:55:16.994319 25764 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786517687384214-19665-0/minicluster-data/master-0/data/instance:
uuid: "05bc58ddb1bb40568bde5f2fb78b3f52"
format_stamp: "Formatted at 2026-08-12 06:55:16 on dist-test-slave-65mx"
server_key: "e749076b4ea32a16eb9365e681b1c16d"
server_key_iv: "2640b813a258bb44be6cfe5acc87222d"
server_key_version: "encryptionkey@0"
I20260812 06:55:16.994915 25764 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786517687384214-19665-0/minicluster-data/master-0/wal/instance:
uuid: "05bc58ddb1bb40568bde5f2fb78b3f52"
format_stamp: "Formatted at 2026-08-12 06:55:16 on dist-test-slave-65mx"
server_key: "e749076b4ea32a16eb9365e681b1c16d"
server_key_iv: "2640b813a258bb44be6cfe5acc87222d"
server_key_version: "encryptionkey@0"
I20260812 06:55:16.998870 25764 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.002s
I20260812 06:55:17.001475 25781 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 06:55:17.002818 25764 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260812 06:55:17.002978 25764 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786517687384214-19665-0/minicluster-data/master-0/data,/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786517687384214-19665-0/minicluster-data/master-0/wal
uuid: "05bc58ddb1bb40568bde5f2fb78b3f52"
format_stamp: "Formatted at 2026-08-12 06:55:16 on dist-test-slave-65mx"
server_key: "e749076b4ea32a16eb9365e681b1c16d"
server_key_iv: "2640b813a258bb44be6cfe5acc87222d"
server_key_version: "encryptionkey@0"
I20260812 06:55:17.003136 25764 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786517687384214-19665-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786517687384214-19665-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786517687384214-19665-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:55:17.020709 25764 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 06:55:17.021518 25764 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 06:55:17.021746 25764 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:55:17.031364 25764 rpc_server.cc:307] RPC server started. Bound to: 127.19.52.126:40473
I20260812 06:55:17.031581 25844 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.52.126:40473 every 8 connection(s)
I20260812 06:55:17.032755 25764 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786517687384214-19665-0/minicluster-data/master-0/data/info.pb
I20260812 06:55:17.036621 25845 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 06:55:17.040498 19665 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu as pid 25764
I20260812 06:55:17.040674 19665 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786517687384214-19665-0/minicluster-data/master-0/wal/instance
I20260812 06:55:17.041002 19665 external_mini_cluster.cc:1428] Setting key cd632d416489003cc1b94fccab9beb47
I20260812 06:55:17.045217 25845 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 05bc58ddb1bb40568bde5f2fb78b3f52: Bootstrap starting.
I20260812 06:55:17.047757 25845 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 05bc58ddb1bb40568bde5f2fb78b3f52: Neither blocks nor log segments found. Creating new log.
I20260812 06:55:17.048632 25845 log.cc:824] T 00000000000000000000000000000000 P 05bc58ddb1bb40568bde5f2fb78b3f52: Log is configured to *not* fsync() on all Append() calls
I20260812 06:55:17.051186 25845 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 05bc58ddb1bb40568bde5f2fb78b3f52: No bootstrap required, opened a new log
I20260812 06:55:17.052248 25848 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:17.042993 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57040 (local address 127.19.52.126:40473)
0812 06:55:17.043957 (+ 964us) server_negotiation.cc:206] Beginning negotiation
0812 06:55:17.043975 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0812 06:55:17.044016 (+ 41us) server_negotiation.cc:407] Connection header received
0812 06:55:17.044821 (+ 805us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:55:17.044841 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:55:17.045281 (+ 440us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:55:17.045724 (+ 443us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:55:17.046220 (+ 496us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:17.047224 (+ 1004us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:17.048246 (+ 1022us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:17.048868 (+ 622us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:17.049056 (+ 188us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:55:17.049983 (+ 927us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:55:17.049989 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:55:17.050004 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 06:55:17.050543 (+ 539us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:55:17.050644 (+ 101us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 06:55:17.050668 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:55:17.050768 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0812 06:55:17.051001 (+ 233us) server_negotiation.cc:299] Negotiation successful
0812 06:55:17.051366 (+ 365us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":550,"thread_start_us":201,"threads_started":1}
I20260812 06:55:17.056314 19665 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 06:55:17.056869 25845 raft_consensus.cc:348] T 00000000000000000000000000000000 P 05bc58ddb1bb40568bde5f2fb78b3f52 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "05bc58ddb1bb40568bde5f2fb78b3f52" member_type: VOTER last_known_addr { host: "127.19.52.126" port: 40473 } }
I20260812 06:55:17.057313 25845 raft_consensus.cc:374] T 00000000000000000000000000000000 P 05bc58ddb1bb40568bde5f2fb78b3f52 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 06:55:17.057525 25845 raft_consensus.cc:729] T 00000000000000000000000000000000 P 05bc58ddb1bb40568bde5f2fb78b3f52 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 05bc58ddb1bb40568bde5f2fb78b3f52, State: Initialized, Role: FOLLOWER
I20260812 06:55:17.058122 25845 consensus_queue.cc:260] T 00000000000000000000000000000000 P 05bc58ddb1bb40568bde5f2fb78b3f52 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "05bc58ddb1bb40568bde5f2fb78b3f52" member_type: VOTER last_known_addr { host: "127.19.52.126" port: 40473 } }
I20260812 06:55:17.058311 25845 raft_consensus.cc:388] T 00000000000000000000000000000000 P 05bc58ddb1bb40568bde5f2fb78b3f52 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 06:55:17.058487 25845 raft_consensus.cc:482] T 00000000000000000000000000000000 P 05bc58ddb1bb40568bde5f2fb78b3f52 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 06:55:17.058656 25845 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 05bc58ddb1bb40568bde5f2fb78b3f52 [term 0 FOLLOWER]: Advancing to term 1
I20260812 06:55:17.060019 25845 raft_consensus.cc:504] T 00000000000000000000000000000000 P 05bc58ddb1bb40568bde5f2fb78b3f52 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "05bc58ddb1bb40568bde5f2fb78b3f52" member_type: VOTER last_known_addr { host: "127.19.52.126" port: 40473 } }
I20260812 06:55:17.060388 25845 leader_election.cc:302] T 00000000000000000000000000000000 P 05bc58ddb1bb40568bde5f2fb78b3f52 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 05bc58ddb1bb40568bde5f2fb78b3f52; no voters:
I20260812 06:55:17.060938 25845 leader_election.cc:288] T 00000000000000000000000000000000 P 05bc58ddb1bb40568bde5f2fb78b3f52 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 06:55:17.061221 25850 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 05bc58ddb1bb40568bde5f2fb78b3f52 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 06:55:17.061501 25850 raft_consensus.cc:686] T 00000000000000000000000000000000 P 05bc58ddb1bb40568bde5f2fb78b3f52 [term 1 LEADER]: Becoming Leader. State: Replica: 05bc58ddb1bb40568bde5f2fb78b3f52, State: Running, Role: LEADER
I20260812 06:55:17.061859 25850 consensus_queue.cc:237] T 00000000000000000000000000000000 P 05bc58ddb1bb40568bde5f2fb78b3f52 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "05bc58ddb1bb40568bde5f2fb78b3f52" member_type: VOTER last_known_addr { host: "127.19.52.126" port: 40473 } }
I20260812 06:55:17.062407 25845 sys_catalog.cc:564] T 00000000000000000000000000000000 P 05bc58ddb1bb40568bde5f2fb78b3f52 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 06:55:17.063868 25851 sys_catalog.cc:455] T 00000000000000000000000000000000 P 05bc58ddb1bb40568bde5f2fb78b3f52 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "05bc58ddb1bb40568bde5f2fb78b3f52" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "05bc58ddb1bb40568bde5f2fb78b3f52" member_type: VOTER last_known_addr { host: "127.19.52.126" port: 40473 } } }
I20260812 06:55:17.064095 25851 sys_catalog.cc:458] T 00000000000000000000000000000000 P 05bc58ddb1bb40568bde5f2fb78b3f52 [sys.catalog]: This master's current role is: LEADER
I20260812 06:55:17.064347 25852 sys_catalog.cc:455] T 00000000000000000000000000000000 P 05bc58ddb1bb40568bde5f2fb78b3f52 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 05bc58ddb1bb40568bde5f2fb78b3f52. Latest consensus state: current_term: 1 leader_uuid: "05bc58ddb1bb40568bde5f2fb78b3f52" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "05bc58ddb1bb40568bde5f2fb78b3f52" member_type: VOTER last_known_addr { host: "127.19.52.126" port: 40473 } } }
I20260812 06:55:17.064489 25852 sys_catalog.cc:458] T 00000000000000000000000000000000 P 05bc58ddb1bb40568bde5f2fb78b3f52 [sys.catalog]: This master's current role is: LEADER
I20260812 06:55:17.064597 25862 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 06:55:17.067574 25862 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 06:55:17.073990 25862 catalog_manager.cc:1347] Generated new cluster ID: 89b24439a2ae451f9344065db2c14fec
I20260812 06:55:17.074098 25862 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 06:55:17.092163 25862 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 06:55:17.093304 25862 catalog_manager.cc:1504] Loading token signing keys...
I20260812 06:55:17.103080 25862 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 05bc58ddb1bb40568bde5f2fb78b3f52: Generated new TSK 0
I20260812 06:55:17.104158 25862 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260812 06:55:17.133931 25881 webserver.cc:533] Webserver: sslize failed: error:14094418:SSL routines:ssl3_read_bytes:tlsv1 alert unknown ca
E20260812 06:55:17.134037 25881 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
W20260812 06:55:17.135010 25879 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 06:55:17.112891 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.126:40473 (local address 127.0.0.1:57056)
0812 06:55:17.113255 (+ 364us) negotiation.cc:107] Waiting for socket to connect
0812 06:55:17.113273 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0812 06:55:17.113446 (+ 173us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:55:17.113820 (+ 374us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:55:17.113823 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:55:17.113832 (+ 9us) client_negotiation.cc:189] Negotiated authn=JWT
0812 06:55:17.113960 (+ 128us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:55:17.113968 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:55:17.114760 (+ 792us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:55:17.114767 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:55:17.115879 (+ 1112us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:55:17.115892 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:55:17.116375 (+ 483us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:55:17.116379 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:55:17.116477 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:55:17.116508 (+ 31us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0812 06:55:17.134606 (+ 18098us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:44423/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0812 06:55:17.134809 (+ 203us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.19.52.126:40473: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:44423/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"client-negotiator.queue_time_us":267,"thread_start_us":137,"threads_started":1}
I20260812 06:55:17.135025 25848 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:17.113067 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57056 (local address 127.19.52.126:40473)
0812 06:55:17.113192 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0812 06:55:17.113197 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 06:55:17.113366 (+ 169us) server_negotiation.cc:407] Connection header received
0812 06:55:17.113628 (+ 262us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:55:17.113631 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:55:17.113686 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:55:17.113790 (+ 104us) server_negotiation.cc:226] Negotiated authn=JWT
0812 06:55:17.114081 (+ 291us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:17.114636 (+ 555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:17.116028 (+ 1392us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:17.116251 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:17.116341 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:55:17.116737 (+ 396us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0812 06:55:17.134375 (+ 17638us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:44423/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0812 06:55:17.134747 (+ 372us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:57056: Failed to load JWKS: Error downloading JWKS from 'https://localhost:44423/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"server-negotiator.queue_time_us":37}
W20260812 06:55:17.135131 25848 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:57056: Failed to load JWKS: Error downloading JWKS from 'https://localhost:44423/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
I20260812 06:55:17.135602 19665 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu with pid 25764
2026-08-12T06:55:17Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithInvalidCert (348 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithUntrustedCert
I20260812 06:55:17.179101 19665 mini_oidc.cc:152] Starting JWKS server
I20260812 06:55:17.180029 19665 webserver.cc:460] Webserver started at https://127.0.0.1:39711/ using document root <none> and password file <none>
I20260812 06:55:17.180147 19665 mini_oidc.cc:189] Starting OIDC Discovery server
I20260812 06:55:17.180446 19665 webserver.cc:460] Webserver started at http://127.0.0.1:38779/ using document root <none> and password file <none>
2026-08-12T06:55:17Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T06:55:17Z Disabled control of system clock
I20260812 06:55:17.197144 19665 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu
/tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786517687384214-19665-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786517687384214-19665-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786517687384214-19665-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786517687384214-19665-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.52.126:43907
--webserver_interface=127.19.52.126
--webserver_port=0
--builtin_ntp_servers=127.19.52.84:37831
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.19.52.126:43907
--enable_jwt_token_auth=true
--jwks_url=https://localhost:39711/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260812 06:55:17.318969 25887 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:55:17.319265 25887 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:55:17.319307 25887 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 06:55:17.323637 25887 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 06:55:17.323737 25887 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 06:55:17.323756 25887 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 06:55:17.323778 25887 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 06:55:17.323796 25887 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260812 06:55:17.323854 25887 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260812 06:55:17.323896 25887 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:39711/jwks.json
I20260812 06:55:17.329717 25887 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.52.84:37831
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786517687384214-19665-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786517687384214-19665-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.19.52.126:43907
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.52.126:43907
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:39711/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786517687384214-19665-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.19.52.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25887
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786517687384214-19665-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 06:15:54 UTC on ccycloud.quasar-rnfzce.root.comops.site
build id 4581989
I20260812 06:55:17.331004 25887 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:55:17.332825 25887 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 06:55:17.342166 25893 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:55:17.342167 25896 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:55:17.342229 25887 server_base.cc:1034] running on GCE node
W20260812 06:55:17.342172 25894 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:55:17.343146 25887 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:55:17.344328 25887 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:55:17.345587 25887 hybrid_clock.cc:648] HybridClock initialized: now 1786517717345558 us; error 53 us; skew 500 ppm
I20260812 06:55:17.347883 25887 webserver.cc:460] Webserver started at http://127.19.52.126:35803/ using document root <none> and password file <none>
I20260812 06:55:17.348598 25887 fs_manager.cc:359] Metadata directory not provided
I20260812 06:55:17.348675 25887 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:55:17.348915 25887 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 06:55:17.351293 25887 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786517687384214-19665-0/minicluster-data/master-0/data/instance:
uuid: "bcf6ae8390184dfc8e841bb7feb5ca91"
format_stamp: "Formatted at 2026-08-12 06:55:17 on dist-test-slave-65mx"
server_key: "d90275ae696fb22e13de6a7f01865a79"
server_key_iv: "fd74250047134d26caaf117d393c13bb"
server_key_version: "encryptionkey@0"
I20260812 06:55:17.351954 25887 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786517687384214-19665-0/minicluster-data/master-0/wal/instance:
uuid: "bcf6ae8390184dfc8e841bb7feb5ca91"
format_stamp: "Formatted at 2026-08-12 06:55:17 on dist-test-slave-65mx"
server_key: "d90275ae696fb22e13de6a7f01865a79"
server_key_iv: "fd74250047134d26caaf117d393c13bb"
server_key_version: "encryptionkey@0"
I20260812 06:55:17.356201 25887 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260812 06:55:17.359367 25902 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 06:55:17.361333 25887 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 06:55:17.361527 25887 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786517687384214-19665-0/minicluster-data/master-0/data,/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786517687384214-19665-0/minicluster-data/master-0/wal
uuid: "bcf6ae8390184dfc8e841bb7feb5ca91"
format_stamp: "Formatted at 2026-08-12 06:55:17 on dist-test-slave-65mx"
server_key: "d90275ae696fb22e13de6a7f01865a79"
server_key_iv: "fd74250047134d26caaf117d393c13bb"
server_key_version: "encryptionkey@0"
I20260812 06:55:17.361704 25887 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786517687384214-19665-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786517687384214-19665-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786517687384214-19665-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:55:17.397761 25887 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 06:55:17.398577 25887 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 06:55:17.398795 25887 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:55:17.406447 25887 rpc_server.cc:307] RPC server started. Bound to: 127.19.52.126:43907
I20260812 06:55:17.406530 25964 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.52.126:43907 every 8 connection(s)
I20260812 06:55:17.407860 25887 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786517687384214-19665-0/minicluster-data/master-0/data/info.pb
I20260812 06:55:17.412566 25965 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 06:55:17.416188 19665 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu as pid 25887
I20260812 06:55:17.416326 19665 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786517687384214-19665-0/minicluster-data/master-0/wal/instance
I20260812 06:55:17.416684 19665 external_mini_cluster.cc:1428] Setting key f3285f844345980439f440552bac7053
I20260812 06:55:17.419898 25965 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bcf6ae8390184dfc8e841bb7feb5ca91: Bootstrap starting.
I20260812 06:55:17.422281 25965 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P bcf6ae8390184dfc8e841bb7feb5ca91: Neither blocks nor log segments found. Creating new log.
I20260812 06:55:17.423065 25965 log.cc:824] T 00000000000000000000000000000000 P bcf6ae8390184dfc8e841bb7feb5ca91: Log is configured to *not* fsync() on all Append() calls
I20260812 06:55:17.426066 25965 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bcf6ae8390184dfc8e841bb7feb5ca91: No bootstrap required, opened a new log
I20260812 06:55:17.427671 25968 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:17.418194 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39672 (local address 127.19.52.126:43907)
0812 06:55:17.418955 (+ 761us) server_negotiation.cc:206] Beginning negotiation
0812 06:55:17.418970 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0812 06:55:17.419009 (+ 39us) server_negotiation.cc:407] Connection header received
0812 06:55:17.419648 (+ 639us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:55:17.419665 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:55:17.420160 (+ 495us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:55:17.420531 (+ 371us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:55:17.421028 (+ 497us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:17.421997 (+ 969us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:17.422998 (+ 1001us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:17.423533 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:17.423712 (+ 179us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:55:17.424745 (+ 1033us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:55:17.424752 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:55:17.424768 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 06:55:17.425318 (+ 550us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:55:17.425399 (+ 81us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 06:55:17.425428 (+ 29us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:55:17.425550 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0812 06:55:17.425770 (+ 220us) server_negotiation.cc:299] Negotiation successful
0812 06:55:17.426133 (+ 363us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":370,"thread_start_us":156,"threads_started":1}
I20260812 06:55:17.430951 25965 raft_consensus.cc:348] T 00000000000000000000000000000000 P bcf6ae8390184dfc8e841bb7feb5ca91 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bcf6ae8390184dfc8e841bb7feb5ca91" member_type: VOTER last_known_addr { host: "127.19.52.126" port: 43907 } }
I20260812 06:55:17.431246 25965 raft_consensus.cc:374] T 00000000000000000000000000000000 P bcf6ae8390184dfc8e841bb7feb5ca91 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 06:55:17.431304 25965 raft_consensus.cc:729] T 00000000000000000000000000000000 P bcf6ae8390184dfc8e841bb7feb5ca91 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bcf6ae8390184dfc8e841bb7feb5ca91, State: Initialized, Role: FOLLOWER
I20260812 06:55:17.431645 19665 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 06:55:17.431783 25965 consensus_queue.cc:260] T 00000000000000000000000000000000 P bcf6ae8390184dfc8e841bb7feb5ca91 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bcf6ae8390184dfc8e841bb7feb5ca91" member_type: VOTER last_known_addr { host: "127.19.52.126" port: 43907 } }
I20260812 06:55:17.431964 25965 raft_consensus.cc:388] T 00000000000000000000000000000000 P bcf6ae8390184dfc8e841bb7feb5ca91 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 06:55:17.432045 25965 raft_consensus.cc:482] T 00000000000000000000000000000000 P bcf6ae8390184dfc8e841bb7feb5ca91 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 06:55:17.432168 25965 raft_consensus.cc:3037] T 00000000000000000000000000000000 P bcf6ae8390184dfc8e841bb7feb5ca91 [term 0 FOLLOWER]: Advancing to term 1
I20260812 06:55:17.433492 25965 raft_consensus.cc:504] T 00000000000000000000000000000000 P bcf6ae8390184dfc8e841bb7feb5ca91 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bcf6ae8390184dfc8e841bb7feb5ca91" member_type: VOTER last_known_addr { host: "127.19.52.126" port: 43907 } }
I20260812 06:55:17.433881 25965 leader_election.cc:302] T 00000000000000000000000000000000 P bcf6ae8390184dfc8e841bb7feb5ca91 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: bcf6ae8390184dfc8e841bb7feb5ca91; no voters:
I20260812 06:55:17.434266 25965 leader_election.cc:288] T 00000000000000000000000000000000 P bcf6ae8390184dfc8e841bb7feb5ca91 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 06:55:17.434408 25970 raft_consensus.cc:2781] T 00000000000000000000000000000000 P bcf6ae8390184dfc8e841bb7feb5ca91 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 06:55:17.434688 25970 raft_consensus.cc:686] T 00000000000000000000000000000000 P bcf6ae8390184dfc8e841bb7feb5ca91 [term 1 LEADER]: Becoming Leader. State: Replica: bcf6ae8390184dfc8e841bb7feb5ca91, State: Running, Role: LEADER
I20260812 06:55:17.435022 25970 consensus_queue.cc:237] T 00000000000000000000000000000000 P bcf6ae8390184dfc8e841bb7feb5ca91 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bcf6ae8390184dfc8e841bb7feb5ca91" member_type: VOTER last_known_addr { host: "127.19.52.126" port: 43907 } }
I20260812 06:55:17.435621 25965 sys_catalog.cc:564] T 00000000000000000000000000000000 P bcf6ae8390184dfc8e841bb7feb5ca91 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 06:55:17.436918 25972 sys_catalog.cc:455] T 00000000000000000000000000000000 P bcf6ae8390184dfc8e841bb7feb5ca91 [sys.catalog]: SysCatalogTable state changed. Reason: New leader bcf6ae8390184dfc8e841bb7feb5ca91. Latest consensus state: current_term: 1 leader_uuid: "bcf6ae8390184dfc8e841bb7feb5ca91" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bcf6ae8390184dfc8e841bb7feb5ca91" member_type: VOTER last_known_addr { host: "127.19.52.126" port: 43907 } } }
I20260812 06:55:17.437081 25972 sys_catalog.cc:458] T 00000000000000000000000000000000 P bcf6ae8390184dfc8e841bb7feb5ca91 [sys.catalog]: This master's current role is: LEADER
I20260812 06:55:17.437577 25981 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 06:55:17.438228 25971 sys_catalog.cc:455] T 00000000000000000000000000000000 P bcf6ae8390184dfc8e841bb7feb5ca91 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "bcf6ae8390184dfc8e841bb7feb5ca91" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bcf6ae8390184dfc8e841bb7feb5ca91" member_type: VOTER last_known_addr { host: "127.19.52.126" port: 43907 } } }
I20260812 06:55:17.438352 25971 sys_catalog.cc:458] T 00000000000000000000000000000000 P bcf6ae8390184dfc8e841bb7feb5ca91 [sys.catalog]: This master's current role is: LEADER
I20260812 06:55:17.441298 25981 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 06:55:17.447757 25981 catalog_manager.cc:1347] Generated new cluster ID: afe268d2a4e04e7b8a824a2cfd389ef2
I20260812 06:55:17.447862 25981 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 06:55:17.461998 25981 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 06:55:17.463398 25981 catalog_manager.cc:1504] Loading token signing keys...
I20260812 06:55:17.478044 25981 catalog_manager.cc:5868] T 00000000000000000000000000000000 P bcf6ae8390184dfc8e841bb7feb5ca91: Generated new TSK 0
I20260812 06:55:17.478987 25981 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 06:55:17.499575 25968 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:17.494183 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39678 (local address 127.19.52.126:43907)
0812 06:55:17.494329 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0812 06:55:17.494334 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 06:55:17.494536 (+ 202us) server_negotiation.cc:407] Connection header received
0812 06:55:17.494789 (+ 253us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:55:17.494792 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:55:17.494845 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:55:17.494978 (+ 133us) server_negotiation.cc:226] Negotiated authn=JWT
0812 06:55:17.495258 (+ 280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:17.495837 (+ 579us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:17.496965 (+ 1128us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:17.497242 (+ 277us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:17.497362 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:55:17.497712 (+ 350us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0812 06:55:17.499173 (+ 1461us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:39711/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0812 06:55:17.499415 (+ 242us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:39678: Failed to load JWKS: Error downloading JWKS from 'https://localhost:39711/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"server-negotiator.queue_time_us":66}
W20260812 06:55:17.499655 25968 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:39678: Failed to load JWKS: Error downloading JWKS from 'https://localhost:39711/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
W20260812 06:55:17.501571 26004 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 06:55:17.494003 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.126:43907 (local address 127.0.0.1:39678)
0812 06:55:17.494335 (+ 332us) negotiation.cc:107] Waiting for socket to connect
0812 06:55:17.494351 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0812 06:55:17.494614 (+ 263us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:55:17.494981 (+ 367us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:55:17.494984 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:55:17.494993 (+ 9us) client_negotiation.cc:189] Negotiated authn=JWT
0812 06:55:17.495141 (+ 148us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:55:17.495147 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:55:17.495961 (+ 814us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:55:17.495965 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:55:17.496794 (+ 829us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:55:17.496805 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:55:17.497382 (+ 577us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:55:17.497387 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:55:17.497484 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:55:17.497509 (+ 25us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0812 06:55:17.499359 (+ 1850us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:39711/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0812 06:55:17.501369 (+ 2010us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.19.52.126:43907: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:39711/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"client-negotiator.queue_time_us":239,"mutex_wait_us":1766,"thread_start_us":117,"threads_started":1}
I20260812 06:55:17.501994 19665 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu with pid 25887
E20260812 06:55:17.513892 26005 webserver.cc:533] Webserver: sslize failed:
E20260812 06:55:17.514007 26005 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
2026-08-12T06:55:17Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithUntrustedCert (367 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS
2026-08-12T06:55:17Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T06:55:17Z Disabled control of system clock
I20260812 06:55:17.558038 19665 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu
/tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786517687384214-19665-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786517687384214-19665-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786517687384214-19665-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786517687384214-19665-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.52.126:36917
--webserver_interface=127.19.52.126
--webserver_port=0
--builtin_ntp_servers=127.19.52.84:38803
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.19.52.126:36917
--enable_jwt_token_auth=true
--jwks_url=default.url
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786517687384214-19665-0/testchainca.cert with env {}
W20260812 06:55:17.686174 26009 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:55:17.686489 26009 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:55:17.686550 26009 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 06:55:17.691143 26009 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 06:55:17.691250 26009 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 06:55:17.691268 26009 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 06:55:17.691287 26009 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 06:55:17.691304 26009 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260812 06:55:17.691321 26009 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260812 06:55:17.691335 26009 flags.cc:425] Enabled experimental flag: --jwks_url=default.url
I20260812 06:55:17.697419 26009 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.52.84:38803
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786517687384214-19665-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786517687384214-19665-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.19.52.126:36917
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.52.126:36917
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=default.url
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786517687384214-19665-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.19.52.126
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786517687384214-19665-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26009
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786517687384214-19665-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 06:15:54 UTC on ccycloud.quasar-rnfzce.root.comops.site
build id 4581989
I20260812 06:55:17.698592 26009 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:55:17.699630 26009 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 06:55:17.708806 26018 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:55:17.708840 26009 server_base.cc:1034] running on GCE node
W20260812 06:55:17.708806 26015 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:55:17.709208 26016 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:55:17.709858 26009 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:55:17.710986 26009 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:55:17.712173 26009 hybrid_clock.cc:648] HybridClock initialized: now 1786517717712140 us; error 45 us; skew 500 ppm
I20260812 06:55:17.714248 26009 webserver.cc:460] Webserver started at http://127.19.52.126:46297/ using document root <none> and password file <none>
I20260812 06:55:17.714816 26009 fs_manager.cc:359] Metadata directory not provided
I20260812 06:55:17.714884 26009 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:55:17.715068 26009 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 06:55:17.717371 26009 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786517687384214-19665-0/minicluster-data/master-0/data/instance:
uuid: "c3deae176a5c4f0b8451638367739856"
format_stamp: "Formatted at 2026-08-12 06:55:17 on dist-test-slave-65mx"
server_key: "2580a58004e1c1343554f6f774c57b1f"
server_key_iv: "848bffd2aa33561606672d9322382a1c"
server_key_version: "encryptionkey@0"
I20260812 06:55:17.717955 26009 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786517687384214-19665-0/minicluster-data/master-0/wal/instance:
uuid: "c3deae176a5c4f0b8451638367739856"
format_stamp: "Formatted at 2026-08-12 06:55:17 on dist-test-slave-65mx"
server_key: "2580a58004e1c1343554f6f774c57b1f"
server_key_iv: "848bffd2aa33561606672d9322382a1c"
server_key_version: "encryptionkey@0"
I20260812 06:55:17.722208 26009 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260812 06:55:17.725096 26026 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 06:55:17.726655 26009 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 06:55:17.726785 26009 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786517687384214-19665-0/minicluster-data/master-0/data,/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786517687384214-19665-0/minicluster-data/master-0/wal
uuid: "c3deae176a5c4f0b8451638367739856"
format_stamp: "Formatted at 2026-08-12 06:55:17 on dist-test-slave-65mx"
server_key: "2580a58004e1c1343554f6f774c57b1f"
server_key_iv: "848bffd2aa33561606672d9322382a1c"
server_key_version: "encryptionkey@0"
I20260812 06:55:17.727032 26009 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786517687384214-19665-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786517687384214-19665-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786517687384214-19665-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:55:17.746414 26009 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 06:55:17.747429 26009 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 06:55:17.747646 26009 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:55:17.756330 26009 rpc_server.cc:307] RPC server started. Bound to: 127.19.52.126:36917
I20260812 06:55:17.756462 26090 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.52.126:36917 every 8 connection(s)
I20260812 06:55:17.757493 26009 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786517687384214-19665-0/minicluster-data/master-0/data/info.pb
I20260812 06:55:17.757835 19665 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu as pid 26009
I20260812 06:55:17.757984 19665 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786517687384214-19665-0/minicluster-data/master-0/wal/instance
I20260812 06:55:17.758388 19665 external_mini_cluster.cc:1428] Setting key 0faa8faa2ecbeb1e1f7edcdd5eef5135
I20260812 06:55:17.761636 26091 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 06:55:17.768029 26091 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c3deae176a5c4f0b8451638367739856: Bootstrap starting.
I20260812 06:55:17.770123 26094 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:17.759909 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53738 (local address 127.19.52.126:36917)
0812 06:55:17.761081 (+ 1172us) server_negotiation.cc:206] Beginning negotiation
0812 06:55:17.761108 (+ 27us) server_negotiation.cc:399] Waiting for connection header
0812 06:55:17.761180 (+ 72us) server_negotiation.cc:407] Connection header received
0812 06:55:17.762069 (+ 889us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:55:17.762090 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:55:17.762538 (+ 448us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:55:17.762918 (+ 380us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:55:17.763439 (+ 521us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:17.764507 (+ 1068us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:17.765518 (+ 1011us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:17.766034 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:17.766222 (+ 188us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:55:17.768049 (+ 1827us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:55:17.768055 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:55:17.768072 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 06:55:17.768644 (+ 572us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:55:17.768712 (+ 68us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 06:55:17.768736 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:55:17.768849 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0812 06:55:17.769006 (+ 157us) server_negotiation.cc:299] Negotiation successful
0812 06:55:17.769309 (+ 303us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":303,"thread_start_us":151,"threads_started":1}
I20260812 06:55:17.771332 26091 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c3deae176a5c4f0b8451638367739856: Neither blocks nor log segments found. Creating new log.
I20260812 06:55:17.772274 26091 log.cc:824] T 00000000000000000000000000000000 P c3deae176a5c4f0b8451638367739856: Log is configured to *not* fsync() on all Append() calls
I20260812 06:55:17.774574 19665 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 06:55:17.775326 26091 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c3deae176a5c4f0b8451638367739856: No bootstrap required, opened a new log
I20260812 06:55:17.779311 26091 raft_consensus.cc:348] T 00000000000000000000000000000000 P c3deae176a5c4f0b8451638367739856 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c3deae176a5c4f0b8451638367739856" member_type: VOTER last_known_addr { host: "127.19.52.126" port: 36917 } }
I20260812 06:55:17.779590 26091 raft_consensus.cc:374] T 00000000000000000000000000000000 P c3deae176a5c4f0b8451638367739856 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 06:55:17.779659 26091 raft_consensus.cc:729] T 00000000000000000000000000000000 P c3deae176a5c4f0b8451638367739856 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c3deae176a5c4f0b8451638367739856, State: Initialized, Role: FOLLOWER
I20260812 06:55:17.780144 26091 consensus_queue.cc:260] T 00000000000000000000000000000000 P c3deae176a5c4f0b8451638367739856 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c3deae176a5c4f0b8451638367739856" member_type: VOTER last_known_addr { host: "127.19.52.126" port: 36917 } }
I20260812 06:55:17.780280 26091 raft_consensus.cc:388] T 00000000000000000000000000000000 P c3deae176a5c4f0b8451638367739856 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 06:55:17.780400 26091 raft_consensus.cc:482] T 00000000000000000000000000000000 P c3deae176a5c4f0b8451638367739856 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 06:55:17.780504 26091 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c3deae176a5c4f0b8451638367739856 [term 0 FOLLOWER]: Advancing to term 1
I20260812 06:55:17.781654 26091 raft_consensus.cc:504] T 00000000000000000000000000000000 P c3deae176a5c4f0b8451638367739856 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c3deae176a5c4f0b8451638367739856" member_type: VOTER last_known_addr { host: "127.19.52.126" port: 36917 } }
I20260812 06:55:17.782045 26091 leader_election.cc:302] T 00000000000000000000000000000000 P c3deae176a5c4f0b8451638367739856 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c3deae176a5c4f0b8451638367739856; no voters:
I20260812 06:55:17.782408 26091 leader_election.cc:288] T 00000000000000000000000000000000 P c3deae176a5c4f0b8451638367739856 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 06:55:17.782893 26096 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c3deae176a5c4f0b8451638367739856 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 06:55:17.783318 26096 raft_consensus.cc:686] T 00000000000000000000000000000000 P c3deae176a5c4f0b8451638367739856 [term 1 LEADER]: Becoming Leader. State: Replica: c3deae176a5c4f0b8451638367739856, State: Running, Role: LEADER
I20260812 06:55:17.784008 26096 consensus_queue.cc:237] T 00000000000000000000000000000000 P c3deae176a5c4f0b8451638367739856 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c3deae176a5c4f0b8451638367739856" member_type: VOTER last_known_addr { host: "127.19.52.126" port: 36917 } }
I20260812 06:55:17.784608 26091 sys_catalog.cc:564] T 00000000000000000000000000000000 P c3deae176a5c4f0b8451638367739856 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 06:55:17.786176 26098 sys_catalog.cc:455] T 00000000000000000000000000000000 P c3deae176a5c4f0b8451638367739856 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c3deae176a5c4f0b8451638367739856" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c3deae176a5c4f0b8451638367739856" member_type: VOTER last_known_addr { host: "127.19.52.126" port: 36917 } } }
I20260812 06:55:17.786331 26098 sys_catalog.cc:458] T 00000000000000000000000000000000 P c3deae176a5c4f0b8451638367739856 [sys.catalog]: This master's current role is: LEADER
I20260812 06:55:17.786629 26099 sys_catalog.cc:455] T 00000000000000000000000000000000 P c3deae176a5c4f0b8451638367739856 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c3deae176a5c4f0b8451638367739856. Latest consensus state: current_term: 1 leader_uuid: "c3deae176a5c4f0b8451638367739856" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c3deae176a5c4f0b8451638367739856" member_type: VOTER last_known_addr { host: "127.19.52.126" port: 36917 } } }
I20260812 06:55:17.786723 26099 sys_catalog.cc:458] T 00000000000000000000000000000000 P c3deae176a5c4f0b8451638367739856 [sys.catalog]: This master's current role is: LEADER
I20260812 06:55:17.786772 26109 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 06:55:17.790779 26109 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 06:55:17.797641 26109 catalog_manager.cc:1347] Generated new cluster ID: 2f5c544974d845099ab558dadf338b05
I20260812 06:55:17.797761 26109 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 06:55:17.808326 26109 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 06:55:17.809451 26109 catalog_manager.cc:1504] Loading token signing keys...
I20260812 06:55:17.817617 26109 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c3deae176a5c4f0b8451638367739856: Generated new TSK 0
I20260812 06:55:17.818425 26109 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 06:55:18.265558 26094 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:17.830878 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53752 (local address 127.19.52.126:36917)
0812 06:55:17.831080 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0812 06:55:17.831085 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 06:55:17.831401 (+ 316us) server_negotiation.cc:407] Connection header received
0812 06:55:17.831755 (+ 354us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:55:17.831758 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:55:17.831819 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:55:17.831936 (+ 117us) server_negotiation.cc:226] Negotiated authn=JWT
0812 06:55:17.832397 (+ 461us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:17.833235 (+ 838us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:17.834771 (+ 1536us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:17.835061 (+ 290us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:17.835164 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:55:17.835857 (+ 693us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0812 06:55:18.265121 (+429264us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0812 06:55:18.265427 (+ 306us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:53752: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"server-negotiator.queue_time_us":99}
W20260812 06:55:18.265645 26094 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:53752: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
W20260812 06:55:18.265986 26125 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 06:55:17.830775 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.126:36917 (local address 127.0.0.1:53752)
0812 06:55:17.831297 (+ 522us) negotiation.cc:107] Waiting for socket to connect
0812 06:55:17.831313 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0812 06:55:17.831583 (+ 270us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:55:17.832117 (+ 534us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:55:17.832121 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:55:17.832131 (+ 10us) client_negotiation.cc:189] Negotiated authn=JWT
0812 06:55:17.832262 (+ 131us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:55:17.832269 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:55:17.833560 (+ 1291us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:55:17.833565 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:55:17.834608 (+ 1043us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:55:17.834622 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:55:17.835224 (+ 602us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:55:17.835228 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:55:17.835346 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:55:17.835382 (+ 36us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0812 06:55:18.265409 (+430027us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0812 06:55:18.265622 (+ 213us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.19.52.126:36917: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"client-negotiator.queue_time_us":423,"thread_start_us":153,"threads_started":1}
I20260812 06:55:18.266795 19665 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu with pid 26009
2026-08-12T06:55:18Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS (744 ms)
[ RUN ] SecurityITest.TestWorldReadableKeytab
[ OK ] SecurityITest.TestWorldReadableKeytab (145 ms)
[ RUN ] SecurityITest.TestWorldReadablePrivateKey
[ OK ] SecurityITest.TestWorldReadablePrivateKey (138 ms)
[ RUN ] SecurityITest.TestCorruptKerberosCC
Loading random data
Initializing database '/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786517687384214-19665-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 06:55:18 dist-test-slave-65mx krb5kdc[26141](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 06:55:18 dist-test-slave-65mx krb5kdc[26141](info): set up 2 sockets
Aug 12 06:55:18 dist-test-slave-65mx krb5kdc[26141](info): commencing operation
krb5kdc: starting...
W20260812 06:55:19.704286 19665 mini_kdc.cc:121] Time spent starting KDC: real 1.129s user 0.002s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786517687384214-19665-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786517687384214-19665-0/krb5kdc/test-user.keytab.
Aug 12 06:55:19 dist-test-slave-65mx krb5kdc[26141](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517719, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T06:55:19Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T06:55:19Z Disabled control of system clock
WARNING: no policy specified for kudu/127.19.52.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.52.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.52.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786517687384214-19665-0/krb5kdc/kudu_127.19.52.126.keytab.
Entry for principal kudu/127.19.52.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786517687384214-19665-0/krb5kdc/kudu_127.19.52.126.keytab.
I20260812 06:55:19.848481 19665 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu
/tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786517687384214-19665-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786517687384214-19665-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786517687384214-19665-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786517687384214-19665-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.52.126:40117
--webserver_interface=127.19.52.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786517687384214-19665-0/krb5kdc/kudu_127.19.52.126.keytab
--principal=kudu/127.19.52.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.52.84:33737
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.19.52.126:40117
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786517687384214-19665-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786517687384214-19665-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786517687384214-19665-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 06:55:19.972115 26157 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:55:19.972504 26157 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:55:19.972625 26157 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 06:55:19.976924 26157 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 06:55:19.977025 26157 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 06:55:19.977064 26157 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 06:55:19.977103 26157 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 06:55:19.977140 26157 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 06:55:19.982708 26157 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.52.84:33737
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786517687384214-19665-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786517687384214-19665-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.19.52.126:40117
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786517687384214-19665-0/krb5kdc/kudu_127.19.52.126.keytab
--principal=kudu/127.19.52.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.52.126:40117
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786517687384214-19665-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.19.52.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26157
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786517687384214-19665-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 06:15:54 UTC on ccycloud.quasar-rnfzce.root.comops.site
build id 4581989
I20260812 06:55:19.983940 26157 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:55:19.985070 26157 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 06:55:19.994424 26163 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:55:19.994494 26162 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:55:19.994690 26157 server_base.cc:1034] running on GCE node
W20260812 06:55:19.994547 26165 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:55:19.995659 26157 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:55:19.996928 26157 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:55:19.998108 26157 hybrid_clock.cc:648] HybridClock initialized: now 1786517719998094 us; error 29 us; skew 500 ppm
Aug 12 06:55:19 dist-test-slave-65mx krb5kdc[26141](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517719, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.52.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 06:55:20.004482 26157 init.cc:377] Logged in from keytab as kudu/127.19.52.126@KRBTEST.COM (short username kudu)
I20260812 06:55:20.005546 26157 webserver.cc:460] Webserver started at http://127.19.52.126:38991/ using document root <none> and password file <none>
I20260812 06:55:20.006150 26157 fs_manager.cc:359] Metadata directory not provided
I20260812 06:55:20.006215 26157 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:55:20.006387 26157 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 06:55:20.008469 26157 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786517687384214-19665-0/minicluster-data/master-0/data/instance:
uuid: "eab6ff1e35bd49a58c29f74078bfc3c6"
format_stamp: "Formatted at 2026-08-12 06:55:20 on dist-test-slave-65mx"
server_key: "97d72016bba7f6e79309f57d89891dff"
server_key_iv: "7d34c972c3152e52003de5c89f94f358"
server_key_version: "encryptionkey@0"
I20260812 06:55:20.009091 26157 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786517687384214-19665-0/minicluster-data/master-0/wal/instance:
uuid: "eab6ff1e35bd49a58c29f74078bfc3c6"
format_stamp: "Formatted at 2026-08-12 06:55:20 on dist-test-slave-65mx"
server_key: "97d72016bba7f6e79309f57d89891dff"
server_key_iv: "7d34c972c3152e52003de5c89f94f358"
server_key_version: "encryptionkey@0"
I20260812 06:55:20.013175 26157 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260812 06:55:20.015743 26173 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 06:55:20.017236 26157 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 06:55:20.017382 26157 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786517687384214-19665-0/minicluster-data/master-0/data,/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786517687384214-19665-0/minicluster-data/master-0/wal
uuid: "eab6ff1e35bd49a58c29f74078bfc3c6"
format_stamp: "Formatted at 2026-08-12 06:55:20 on dist-test-slave-65mx"
server_key: "97d72016bba7f6e79309f57d89891dff"
server_key_iv: "7d34c972c3152e52003de5c89f94f358"
server_key_version: "encryptionkey@0"
I20260812 06:55:20.017501 26157 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786517687384214-19665-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786517687384214-19665-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786517687384214-19665-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:55:20.058943 26157 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 06:55:20.062655 26157 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 06:55:20.062915 26157 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:55:20.071801 26157 rpc_server.cc:307] RPC server started. Bound to: 127.19.52.126:40117
I20260812 06:55:20.071867 26230 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.52.126:40117 every 8 connection(s)
I20260812 06:55:20.072934 26157 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786517687384214-19665-0/minicluster-data/master-0/data/info.pb
I20260812 06:55:20.075845 26231 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 06:55:20.077206 19665 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu as pid 26157
I20260812 06:55:20.077358 19665 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786517687384214-19665-0/minicluster-data/master-0/wal/instance
I20260812 06:55:20.077659 19665 external_mini_cluster.cc:1428] Setting key bdfd0a3c918ddccdb923df57a3a337d5
I20260812 06:55:20.083127 26231 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P eab6ff1e35bd49a58c29f74078bfc3c6: Bootstrap starting.
I20260812 06:55:20.085896 26231 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P eab6ff1e35bd49a58c29f74078bfc3c6: Neither blocks nor log segments found. Creating new log.
I20260812 06:55:20.086608 26231 log.cc:824] T 00000000000000000000000000000000 P eab6ff1e35bd49a58c29f74078bfc3c6: Log is configured to *not* fsync() on all Append() calls
Aug 12 06:55:20 dist-test-slave-65mx krb5kdc[26141](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517719, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.19.52.126@KRBTEST.COM
I20260812 06:55:20.089040 26231 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P eab6ff1e35bd49a58c29f74078bfc3c6: No bootstrap required, opened a new log
I20260812 06:55:20.093855 26237 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:20.079024 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45786 (local address 127.19.52.126:40117)
0812 06:55:20.079476 (+ 452us) server_negotiation.cc:206] Beginning negotiation
0812 06:55:20.079504 (+ 28us) server_negotiation.cc:399] Waiting for connection header
0812 06:55:20.079550 (+ 46us) server_negotiation.cc:407] Connection header received
0812 06:55:20.080459 (+ 909us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:55:20.080483 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:55:20.080956 (+ 473us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:55:20.081353 (+ 397us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:55:20.082113 (+ 760us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:20.083553 (+ 1440us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:20.084471 (+ 918us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:20.085197 (+ 726us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:20.085361 (+ 164us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:55:20.088084 (+ 2723us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:55:20.088094 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:55:20.088114 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:55:20.088148 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:55:20.090363 (+ 2215us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:55:20.090846 (+ 483us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:55:20.090852 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:55:20.090858 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:55:20.091020 (+ 162us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:55:20.091317 (+ 297us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:55:20.091322 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:55:20.091327 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:55:20.091943 (+ 616us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:55:20.092145 (+ 202us) server_negotiation.cc:1035] Waiting for connection context
0812 06:55:20.092457 (+ 312us) server_negotiation.cc:299] Negotiation successful
0812 06:55:20.092795 (+ 338us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":286,"thread_start_us":122,"threads_started":1}
I20260812 06:55:20.093982 26231 raft_consensus.cc:348] T 00000000000000000000000000000000 P eab6ff1e35bd49a58c29f74078bfc3c6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eab6ff1e35bd49a58c29f74078bfc3c6" member_type: VOTER last_known_addr { host: "127.19.52.126" port: 40117 } }
I20260812 06:55:20.094260 26231 raft_consensus.cc:374] T 00000000000000000000000000000000 P eab6ff1e35bd49a58c29f74078bfc3c6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 06:55:20.094323 26231 raft_consensus.cc:729] T 00000000000000000000000000000000 P eab6ff1e35bd49a58c29f74078bfc3c6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eab6ff1e35bd49a58c29f74078bfc3c6, State: Initialized, Role: FOLLOWER
I20260812 06:55:20.094807 26231 consensus_queue.cc:260] T 00000000000000000000000000000000 P eab6ff1e35bd49a58c29f74078bfc3c6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eab6ff1e35bd49a58c29f74078bfc3c6" member_type: VOTER last_known_addr { host: "127.19.52.126" port: 40117 } }
I20260812 06:55:20.094975 26231 raft_consensus.cc:388] T 00000000000000000000000000000000 P eab6ff1e35bd49a58c29f74078bfc3c6 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 06:55:20.095047 26231 raft_consensus.cc:482] T 00000000000000000000000000000000 P eab6ff1e35bd49a58c29f74078bfc3c6 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 06:55:20.095153 26231 raft_consensus.cc:3037] T 00000000000000000000000000000000 P eab6ff1e35bd49a58c29f74078bfc3c6 [term 0 FOLLOWER]: Advancing to term 1
I20260812 06:55:20.096227 26231 raft_consensus.cc:504] T 00000000000000000000000000000000 P eab6ff1e35bd49a58c29f74078bfc3c6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eab6ff1e35bd49a58c29f74078bfc3c6" member_type: VOTER last_known_addr { host: "127.19.52.126" port: 40117 } }
I20260812 06:55:20.096657 26231 leader_election.cc:302] T 00000000000000000000000000000000 P eab6ff1e35bd49a58c29f74078bfc3c6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: eab6ff1e35bd49a58c29f74078bfc3c6; no voters:
I20260812 06:55:20.097033 26231 leader_election.cc:288] T 00000000000000000000000000000000 P eab6ff1e35bd49a58c29f74078bfc3c6 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 06:55:20.097301 26241 raft_consensus.cc:2781] T 00000000000000000000000000000000 P eab6ff1e35bd49a58c29f74078bfc3c6 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 06:55:20.097626 26241 raft_consensus.cc:686] T 00000000000000000000000000000000 P eab6ff1e35bd49a58c29f74078bfc3c6 [term 1 LEADER]: Becoming Leader. State: Replica: eab6ff1e35bd49a58c29f74078bfc3c6, State: Running, Role: LEADER
I20260812 06:55:20.097978 26241 consensus_queue.cc:237] T 00000000000000000000000000000000 P eab6ff1e35bd49a58c29f74078bfc3c6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eab6ff1e35bd49a58c29f74078bfc3c6" member_type: VOTER last_known_addr { host: "127.19.52.126" port: 40117 } }
I20260812 06:55:20.098394 26231 sys_catalog.cc:564] T 00000000000000000000000000000000 P eab6ff1e35bd49a58c29f74078bfc3c6 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 06:55:20.099794 26242 sys_catalog.cc:455] T 00000000000000000000000000000000 P eab6ff1e35bd49a58c29f74078bfc3c6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "eab6ff1e35bd49a58c29f74078bfc3c6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eab6ff1e35bd49a58c29f74078bfc3c6" member_type: VOTER last_known_addr { host: "127.19.52.126" port: 40117 } } }
I20260812 06:55:20.099924 26242 sys_catalog.cc:458] T 00000000000000000000000000000000 P eab6ff1e35bd49a58c29f74078bfc3c6 [sys.catalog]: This master's current role is: LEADER
I20260812 06:55:20.100325 26244 sys_catalog.cc:455] T 00000000000000000000000000000000 P eab6ff1e35bd49a58c29f74078bfc3c6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader eab6ff1e35bd49a58c29f74078bfc3c6. Latest consensus state: current_term: 1 leader_uuid: "eab6ff1e35bd49a58c29f74078bfc3c6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eab6ff1e35bd49a58c29f74078bfc3c6" member_type: VOTER last_known_addr { host: "127.19.52.126" port: 40117 } } }
I20260812 06:55:20.100419 26244 sys_catalog.cc:458] T 00000000000000000000000000000000 P eab6ff1e35bd49a58c29f74078bfc3c6 [sys.catalog]: This master's current role is: LEADER
I20260812 06:55:20.100584 26253 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 06:55:20.104038 26253 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 06:55:20.109874 26253 catalog_manager.cc:1347] Generated new cluster ID: 03bc3ef4791d40e19a716b52a5c17d25
I20260812 06:55:20.109962 26253 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 06:55:20.122736 26253 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 06:55:20.123687 26253 catalog_manager.cc:1504] Loading token signing keys...
I20260812 06:55:20.129614 26253 catalog_manager.cc:5868] T 00000000000000000000000000000000 P eab6ff1e35bd49a58c29f74078bfc3c6: Generated new TSK 0
I20260812 06:55:20.130311 26253 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.19.52.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.52.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.52.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786517687384214-19665-0/krb5kdc/kudu_127.19.52.65.keytab.
Entry for principal kudu/127.19.52.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786517687384214-19665-0/krb5kdc/kudu_127.19.52.65.keytab.
I20260812 06:55:20.169576 19665 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu
/tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786517687384214-19665-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786517687384214-19665-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786517687384214-19665-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786517687384214-19665-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.52.65:0
--local_ip_for_outbound_sockets=127.19.52.65
--webserver_interface=127.19.52.65
--webserver_port=0
--tserver_master_addrs=127.19.52.126:40117
--keytab_file=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786517687384214-19665-0/krb5kdc/kudu_127.19.52.65.keytab
--principal=kudu/127.19.52.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.52.84:33737
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786517687384214-19665-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786517687384214-19665-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786517687384214-19665-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 06:55:20.288757 26264 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:55:20.289053 26264 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:55:20.289108 26264 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 06:55:20.293262 26264 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 06:55:20.293334 26264 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 06:55:20.293395 26264 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.52.65
I20260812 06:55:20.299165 26264 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.52.84:33737
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786517687384214-19665-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786517687384214-19665-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786517687384214-19665-0/krb5kdc/kudu_127.19.52.65.keytab
--principal=kudu/127.19.52.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.52.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786517687384214-19665-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.19.52.65
--webserver_port=0
--tserver_master_addrs=127.19.52.126:40117
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26264
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.52.65
--log_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786517687384214-19665-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 06:15:54 UTC on ccycloud.quasar-rnfzce.root.comops.site
build id 4581989
I20260812 06:55:20.300356 26264 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:55:20.301477 26264 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 06:55:20.311542 26270 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:55:20.311542 26269 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:55:20.311641 26273 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:55:20.311903 26264 server_base.cc:1034] running on GCE node
I20260812 06:55:20.312438 26264 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:55:20.313134 26264 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:55:20.314332 26264 hybrid_clock.cc:648] HybridClock initialized: now 1786517720314269 us; error 77 us; skew 500 ppm
Aug 12 06:55:20 dist-test-slave-65mx krb5kdc[26141](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517720, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.52.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 06:55:20.317258 26264 init.cc:377] Logged in from keytab as kudu/127.19.52.65@KRBTEST.COM (short username kudu)
I20260812 06:55:20.318152 26264 webserver.cc:460] Webserver started at http://127.19.52.65:33505/ using document root <none> and password file <none>
I20260812 06:55:20.318719 26264 fs_manager.cc:359] Metadata directory not provided
I20260812 06:55:20.318779 26264 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:55:20.319031 26264 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 06:55:20.321218 26264 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786517687384214-19665-0/minicluster-data/ts-0/data/instance:
uuid: "fc49d071f9414592bddf8b943bacfc13"
format_stamp: "Formatted at 2026-08-12 06:55:20 on dist-test-slave-65mx"
server_key: "5dca884071c43d79bafd79f1d7de865d"
server_key_iv: "3746592fc7011a2f45f48402cc29c8a2"
server_key_version: "encryptionkey@0"
I20260812 06:55:20.321787 26264 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786517687384214-19665-0/minicluster-data/ts-0/wal/instance:
uuid: "fc49d071f9414592bddf8b943bacfc13"
format_stamp: "Formatted at 2026-08-12 06:55:20 on dist-test-slave-65mx"
server_key: "5dca884071c43d79bafd79f1d7de865d"
server_key_iv: "3746592fc7011a2f45f48402cc29c8a2"
server_key_version: "encryptionkey@0"
I20260812 06:55:20.325595 26264 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260812 06:55:20.328166 26281 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 06:55:20.329664 26264 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 06:55:20.329775 26264 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786517687384214-19665-0/minicluster-data/ts-0/data,/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786517687384214-19665-0/minicluster-data/ts-0/wal
uuid: "fc49d071f9414592bddf8b943bacfc13"
format_stamp: "Formatted at 2026-08-12 06:55:20 on dist-test-slave-65mx"
server_key: "5dca884071c43d79bafd79f1d7de865d"
server_key_iv: "3746592fc7011a2f45f48402cc29c8a2"
server_key_version: "encryptionkey@0"
I20260812 06:55:20.329921 26264 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786517687384214-19665-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786517687384214-19665-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786517687384214-19665-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:55:20.348207 26264 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 06:55:20.351760 26264 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 06:55:20.352016 26264 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:55:20.352744 26264 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 06:55:20.353629 26264 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 06:55:20.353677 26264 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 06:55:20.353744 26264 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 06:55:20.353775 26264 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 06:55:20.362934 26264 rpc_server.cc:307] RPC server started. Bound to: 127.19.52.65:36141
I20260812 06:55:20.362962 26415 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.52.65:36141 every 8 connection(s)
I20260812 06:55:20.363951 26264 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786517687384214-19665-0/minicluster-data/ts-0/data/info.pb
I20260812 06:55:20.367877 19665 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu as pid 26264
I20260812 06:55:20.368093 19665 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786517687384214-19665-0/minicluster-data/ts-0/wal/instance
I20260812 06:55:20.368430 19665 external_mini_cluster.cc:1428] Setting key 77e0a26a5bee175390d753dbfdf4ac77
Aug 12 06:55:20 dist-test-slave-65mx krb5kdc[26141](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517720, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.52.65@KRBTEST.COM for kudu/127.19.52.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.19.52.66@KRBTEST.COM; defaulting to no policy
I20260812 06:55:20.381218 26237 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:20.366047 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.52.65:48977 (local address 127.19.52.126:40117)
0812 06:55:20.366221 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0812 06:55:20.366226 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 06:55:20.367175 (+ 949us) server_negotiation.cc:407] Connection header received
0812 06:55:20.369093 (+ 1918us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:55:20.369098 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:55:20.369161 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:55:20.369234 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:55:20.370873 (+ 1639us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:20.371469 (+ 596us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:20.373086 (+ 1617us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:20.373502 (+ 416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:20.373634 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:55:20.376969 (+ 3335us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:55:20.376973 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:55:20.376975 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:55:20.377009 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:55:20.378718 (+ 1709us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:55:20.379364 (+ 646us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:55:20.379367 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:55:20.379369 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:55:20.379453 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:55:20.379836 (+ 383us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:55:20.379839 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:55:20.379840 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:55:20.380044 (+ 204us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:55:20.380200 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0812 06:55:20.380873 (+ 673us) server_negotiation.cc:299] Negotiation successful
0812 06:55:20.381020 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260812 06:55:20.382129 26418 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:20.366478 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.126:40117 (local address 127.19.52.65:48977)
0812 06:55:20.367000 (+ 522us) negotiation.cc:107] Waiting for socket to connect
0812 06:55:20.367051 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0812 06:55:20.368791 (+ 1740us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:55:20.369453 (+ 662us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:55:20.369460 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:55:20.369942 (+ 482us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:55:20.370691 (+ 749us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:55:20.370716 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:55:20.371620 (+ 904us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:55:20.371627 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:55:20.372947 (+ 1320us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:55:20.372958 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:55:20.373740 (+ 782us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:55:20.373745 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:55:20.373946 (+ 201us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:55:20.374629 (+ 683us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 06:55:20.374650 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:55:20.376726 (+ 2076us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 06:55:20.378891 (+ 2165us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:55:20.378899 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:55:20.378926 (+ 27us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:55:20.379233 (+ 307us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:55:20.379562 (+ 329us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:55:20.379566 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:55:20.379568 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:55:20.379728 (+ 160us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:55:20.380201 (+ 473us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 06:55:20.380210 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 06:55:20.380608 (+ 398us) client_negotiation.cc:769] Sending connection context
0812 06:55:20.380815 (+ 207us) client_negotiation.cc:240] Negotiation successful
0812 06:55:20.381187 (+ 372us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":307,"thread_start_us":122,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.52.66@KRBTEST.COM" created.
I20260812 06:55:20.383682 26416 heartbeater.cc:347] Connected to a master server at 127.19.52.126:40117
I20260812 06:55:20.384047 26416 heartbeater.cc:464] Registering TS with master...
I20260812 06:55:20.384696 26416 heartbeater.cc:511] Master 127.19.52.126:40117 requested a full tablet report, sending...
I20260812 06:55:20.386679 26189 ts_manager.cc:194] Registered new tserver with Master: fc49d071f9414592bddf8b943bacfc13 (127.19.52.65:36141)
I20260812 06:55:20.388044 26189 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.52.65@KRBTEST.COM'} at 127.19.52.65:48977
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.52.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786517687384214-19665-0/krb5kdc/kudu_127.19.52.66.keytab.
Entry for principal kudu/127.19.52.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786517687384214-19665-0/krb5kdc/kudu_127.19.52.66.keytab.
I20260812 06:55:20.404621 19665 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu
/tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786517687384214-19665-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786517687384214-19665-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786517687384214-19665-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786517687384214-19665-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.52.66:0
--local_ip_for_outbound_sockets=127.19.52.66
--webserver_interface=127.19.52.66
--webserver_port=0
--tserver_master_addrs=127.19.52.126:40117
--keytab_file=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786517687384214-19665-0/krb5kdc/kudu_127.19.52.66.keytab
--principal=kudu/127.19.52.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.52.84:33737
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786517687384214-19665-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786517687384214-19665-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786517687384214-19665-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 06:55:20.527554 26422 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:55:20.527854 26422 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:55:20.527957 26422 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 06:55:20.534605 26422 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 06:55:20.534693 26422 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 06:55:20.534798 26422 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.52.66
I20260812 06:55:20.543808 26422 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.52.84:33737
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786517687384214-19665-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786517687384214-19665-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786517687384214-19665-0/krb5kdc/kudu_127.19.52.66.keytab
--principal=kudu/127.19.52.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.52.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786517687384214-19665-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.19.52.66
--webserver_port=0
--tserver_master_addrs=127.19.52.126:40117
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26422
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.52.66
--log_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786517687384214-19665-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 06:15:54 UTC on ccycloud.quasar-rnfzce.root.comops.site
build id 4581989
I20260812 06:55:20.545465 26422 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:55:20.546571 26422 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 06:55:20.556677 26427 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:55:20.556685 26428 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:55:20.556681 26430 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:55:20.560724 26422 server_base.cc:1034] running on GCE node
I20260812 06:55:20.561648 26422 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:55:20.562775 26422 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:55:20.564013 26422 hybrid_clock.cc:648] HybridClock initialized: now 1786517720563981 us; error 47 us; skew 500 ppm
Aug 12 06:55:20 dist-test-slave-65mx krb5kdc[26141](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517720, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.52.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 06:55:20.567627 26422 init.cc:377] Logged in from keytab as kudu/127.19.52.66@KRBTEST.COM (short username kudu)
I20260812 06:55:20.568719 26422 webserver.cc:460] Webserver started at http://127.19.52.66:45673/ using document root <none> and password file <none>
I20260812 06:55:20.569344 26422 fs_manager.cc:359] Metadata directory not provided
I20260812 06:55:20.569446 26422 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:55:20.569705 26422 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 06:55:20.572012 26422 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786517687384214-19665-0/minicluster-data/ts-1/data/instance:
uuid: "7efbdbe084b74630919e82496caeeb4c"
format_stamp: "Formatted at 2026-08-12 06:55:20 on dist-test-slave-65mx"
server_key: "42424c1d522f575c42b3f67d31815e0b"
server_key_iv: "7333c585d7a34bc476257b1b6195a269"
server_key_version: "encryptionkey@0"
I20260812 06:55:20.572814 26422 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786517687384214-19665-0/minicluster-data/ts-1/wal/instance:
uuid: "7efbdbe084b74630919e82496caeeb4c"
format_stamp: "Formatted at 2026-08-12 06:55:20 on dist-test-slave-65mx"
server_key: "42424c1d522f575c42b3f67d31815e0b"
server_key_iv: "7333c585d7a34bc476257b1b6195a269"
server_key_version: "encryptionkey@0"
I20260812 06:55:20.576898 26422 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260812 06:55:20.579443 26438 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 06:55:20.580756 26422 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260812 06:55:20.580890 26422 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786517687384214-19665-0/minicluster-data/ts-1/data,/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786517687384214-19665-0/minicluster-data/ts-1/wal
uuid: "7efbdbe084b74630919e82496caeeb4c"
format_stamp: "Formatted at 2026-08-12 06:55:20 on dist-test-slave-65mx"
server_key: "42424c1d522f575c42b3f67d31815e0b"
server_key_iv: "7333c585d7a34bc476257b1b6195a269"
server_key_version: "encryptionkey@0"
I20260812 06:55:20.581004 26422 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786517687384214-19665-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786517687384214-19665-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786517687384214-19665-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:55:20.590811 26422 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 06:55:20.594110 26422 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 06:55:20.594314 26422 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:55:20.594914 26422 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 06:55:20.595772 26422 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 06:55:20.595842 26422 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 06:55:20.595914 26422 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 06:55:20.595965 26422 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 06:55:20.606220 26422 rpc_server.cc:307] RPC server started. Bound to: 127.19.52.66:46285
I20260812 06:55:20.606241 26577 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.52.66:46285 every 8 connection(s)
I20260812 06:55:20.607213 26422 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786517687384214-19665-0/minicluster-data/ts-1/data/info.pb
I20260812 06:55:20.613171 19665 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu as pid 26422
I20260812 06:55:20.613315 19665 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786517687384214-19665-0/minicluster-data/ts-1/wal/instance
I20260812 06:55:20.613603 19665 external_mini_cluster.cc:1428] Setting key 6868663778057d766899dc571bab7421
Aug 12 06:55:20 dist-test-slave-65mx krb5kdc[26141](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517720, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.52.66@KRBTEST.COM for kudu/127.19.52.126@KRBTEST.COM
I20260812 06:55:20.622144 26237 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:20.609305 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.52.66:45785 (local address 127.19.52.126:40117)
0812 06:55:20.609484 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0812 06:55:20.609488 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 06:55:20.610354 (+ 866us) server_negotiation.cc:407] Connection header received
0812 06:55:20.611724 (+ 1370us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:55:20.611727 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:55:20.611775 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:55:20.611891 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:55:20.613335 (+ 1444us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:20.613969 (+ 634us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:20.615337 (+ 1368us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:20.615566 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:20.615675 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:55:20.618396 (+ 2721us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:55:20.618399 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:55:20.618402 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:55:20.618425 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:55:20.619949 (+ 1524us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:55:20.620480 (+ 531us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:55:20.620485 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:55:20.620487 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:55:20.620634 (+ 147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:55:20.621001 (+ 367us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:55:20.621004 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:55:20.621006 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:55:20.621197 (+ 191us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:55:20.621307 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0812 06:55:20.621826 (+ 519us) server_negotiation.cc:299] Negotiation successful
0812 06:55:20.621953 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260812 06:55:20.622970 26580 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:20.609689 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.126:40117 (local address 127.19.52.66:45785)
0812 06:55:20.610196 (+ 507us) negotiation.cc:107] Waiting for socket to connect
0812 06:55:20.610240 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0812 06:55:20.611522 (+ 1282us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:55:20.612151 (+ 629us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:55:20.612157 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:55:20.612527 (+ 370us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:55:20.613186 (+ 659us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:55:20.613202 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:55:20.614097 (+ 895us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:55:20.614100 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:55:20.615214 (+ 1114us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:55:20.615224 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:55:20.615716 (+ 492us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:55:20.615720 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:55:20.615895 (+ 175us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:55:20.616529 (+ 634us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 06:55:20.616585 (+ 56us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:55:20.618216 (+ 1631us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 06:55:20.620069 (+ 1853us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:55:20.620075 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:55:20.620091 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:55:20.620357 (+ 266us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:55:20.620736 (+ 379us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:55:20.620739 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:55:20.620741 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:55:20.620885 (+ 144us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:55:20.621328 (+ 443us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 06:55:20.621335 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 06:55:20.621591 (+ 256us) client_negotiation.cc:769] Sending connection context
0812 06:55:20.621781 (+ 190us) client_negotiation.cc:240] Negotiation successful
0812 06:55:20.622115 (+ 334us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":322,"thread_start_us":115,"threads_started":1}
WARNING: no policy specified for kudu/127.19.52.67@KRBTEST.COM; defaulting to no policy
I20260812 06:55:20.624269 26578 heartbeater.cc:347] Connected to a master server at 127.19.52.126:40117
I20260812 06:55:20.624493 26578 heartbeater.cc:464] Registering TS with master...
I20260812 06:55:20.625046 26578 heartbeater.cc:511] Master 127.19.52.126:40117 requested a full tablet report, sending...
I20260812 06:55:20.626135 26189 ts_manager.cc:194] Registered new tserver with Master: 7efbdbe084b74630919e82496caeeb4c (127.19.52.66:46285)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.52.67@KRBTEST.COM" created.
I20260812 06:55:20.626874 26189 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.52.66@KRBTEST.COM'} at 127.19.52.66:45785
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.52.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786517687384214-19665-0/krb5kdc/kudu_127.19.52.67.keytab.
Entry for principal kudu/127.19.52.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786517687384214-19665-0/krb5kdc/kudu_127.19.52.67.keytab.
I20260812 06:55:20.645942 19665 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu
/tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786517687384214-19665-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786517687384214-19665-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786517687384214-19665-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786517687384214-19665-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.52.67:0
--local_ip_for_outbound_sockets=127.19.52.67
--webserver_interface=127.19.52.67
--webserver_port=0
--tserver_master_addrs=127.19.52.126:40117
--keytab_file=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786517687384214-19665-0/krb5kdc/kudu_127.19.52.67.keytab
--principal=kudu/127.19.52.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.52.84:33737
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786517687384214-19665-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786517687384214-19665-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786517687384214-19665-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 06:55:20.767539 26583 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:55:20.767853 26583 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:55:20.767899 26583 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 06:55:20.772284 26583 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 06:55:20.772377 26583 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 06:55:20.772440 26583 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.52.67
I20260812 06:55:20.778184 26583 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.52.84:33737
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786517687384214-19665-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786517687384214-19665-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786517687384214-19665-0/krb5kdc/kudu_127.19.52.67.keytab
--principal=kudu/127.19.52.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.52.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786517687384214-19665-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.19.52.67
--webserver_port=0
--tserver_master_addrs=127.19.52.126:40117
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26583
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.52.67
--log_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786517687384214-19665-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 06:15:54 UTC on ccycloud.quasar-rnfzce.root.comops.site
build id 4581989
I20260812 06:55:20.779280 26583 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:55:20.780272 26583 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 06:55:20.790746 26590 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:55:20.790746 26591 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:55:20.790895 26583 server_base.cc:1034] running on GCE node
W20260812 06:55:20.790752 26593 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:55:20.791877 26583 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:55:20.792733 26583 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:55:20.793987 26583 hybrid_clock.cc:648] HybridClock initialized: now 1786517720793944 us; error 52 us; skew 500 ppm
Aug 12 06:55:20 dist-test-slave-65mx krb5kdc[26141](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517720, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.52.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 06:55:20.797578 26583 init.cc:377] Logged in from keytab as kudu/127.19.52.67@KRBTEST.COM (short username kudu)
I20260812 06:55:20.798674 26583 webserver.cc:460] Webserver started at http://127.19.52.67:45577/ using document root <none> and password file <none>
I20260812 06:55:20.799376 26583 fs_manager.cc:359] Metadata directory not provided
I20260812 06:55:20.799470 26583 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:55:20.799722 26583 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 06:55:20.801990 26583 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786517687384214-19665-0/minicluster-data/ts-2/data/instance:
uuid: "4f6fff7c528a4c41ad7dbf8b925715d0"
format_stamp: "Formatted at 2026-08-12 06:55:20 on dist-test-slave-65mx"
server_key: "42b63eab597fbebfa1dee2a37c8d861b"
server_key_iv: "a31ac1a1bcc5f1f14f0fbf5289a294ae"
server_key_version: "encryptionkey@0"
I20260812 06:55:20.802598 26583 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786517687384214-19665-0/minicluster-data/ts-2/wal/instance:
uuid: "4f6fff7c528a4c41ad7dbf8b925715d0"
format_stamp: "Formatted at 2026-08-12 06:55:20 on dist-test-slave-65mx"
server_key: "42b63eab597fbebfa1dee2a37c8d861b"
server_key_iv: "a31ac1a1bcc5f1f14f0fbf5289a294ae"
server_key_version: "encryptionkey@0"
I20260812 06:55:20.806612 26583 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260812 06:55:20.809389 26600 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 06:55:20.810765 26583 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260812 06:55:20.810977 26583 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786517687384214-19665-0/minicluster-data/ts-2/data,/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786517687384214-19665-0/minicluster-data/ts-2/wal
uuid: "4f6fff7c528a4c41ad7dbf8b925715d0"
format_stamp: "Formatted at 2026-08-12 06:55:20 on dist-test-slave-65mx"
server_key: "42b63eab597fbebfa1dee2a37c8d861b"
server_key_iv: "a31ac1a1bcc5f1f14f0fbf5289a294ae"
server_key_version: "encryptionkey@0"
I20260812 06:55:20.811123 26583 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786517687384214-19665-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786517687384214-19665-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786517687384214-19665-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:55:20.833669 26583 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 06:55:20.837589 26583 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 06:55:20.837829 26583 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:55:20.838521 26583 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 06:55:20.839485 26583 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 06:55:20.839574 26583 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 06:55:20.839660 26583 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 06:55:20.839708 26583 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 06:55:20.849382 26583 rpc_server.cc:307] RPC server started. Bound to: 127.19.52.67:44377
I20260812 06:55:20.849404 26731 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.52.67:44377 every 8 connection(s)
I20260812 06:55:20.850404 26583 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786517687384214-19665-0/minicluster-data/ts-2/data/info.pb
I20260812 06:55:20.854609 19665 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu as pid 26583
I20260812 06:55:20.854756 19665 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786517687384214-19665-0/minicluster-data/ts-2/wal/instance
I20260812 06:55:20.855075 19665 external_mini_cluster.cc:1428] Setting key 689c1481735594958bf4c88956a7ac31
Aug 12 06:55:20 dist-test-slave-65mx krb5kdc[26141](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517720, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.52.67@KRBTEST.COM for kudu/127.19.52.126@KRBTEST.COM
I20260812 06:55:20.866365 26237 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:20.852287 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.52.67:56701 (local address 127.19.52.126:40117)
0812 06:55:20.852436 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0812 06:55:20.852439 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 06:55:20.853471 (+ 1032us) server_negotiation.cc:407] Connection header received
0812 06:55:20.855113 (+ 1642us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:55:20.855116 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:55:20.855219 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:55:20.855295 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:55:20.856600 (+ 1305us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:20.857324 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:20.858737 (+ 1413us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:20.858988 (+ 251us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:20.859121 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:55:20.861955 (+ 2834us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:55:20.861958 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:55:20.861961 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:55:20.861990 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:55:20.863694 (+ 1704us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:55:20.864283 (+ 589us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:55:20.864286 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:55:20.864288 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:55:20.864377 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:55:20.865034 (+ 657us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:55:20.865037 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:55:20.865038 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:55:20.865304 (+ 266us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:55:20.865529 (+ 225us) server_negotiation.cc:1035] Waiting for connection context
0812 06:55:20.866104 (+ 575us) server_negotiation.cc:299] Negotiation successful
0812 06:55:20.866227 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260812 06:55:20.867261 26734 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:20.852812 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.126:40117 (local address 127.19.52.67:56701)
0812 06:55:20.853302 (+ 490us) negotiation.cc:107] Waiting for socket to connect
0812 06:55:20.853346 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0812 06:55:20.854841 (+ 1495us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:55:20.855505 (+ 664us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:55:20.855511 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:55:20.855866 (+ 355us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:55:20.856449 (+ 583us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:55:20.856465 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:55:20.857461 (+ 996us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:55:20.857465 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:55:20.858599 (+ 1134us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:55:20.858610 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:55:20.859171 (+ 561us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:55:20.859176 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:55:20.859368 (+ 192us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:55:20.860065 (+ 697us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 06:55:20.860083 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:55:20.861743 (+ 1660us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 06:55:20.863824 (+ 2081us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:55:20.863829 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:55:20.863844 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:55:20.864166 (+ 322us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:55:20.864689 (+ 523us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:55:20.864696 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:55:20.864701 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:55:20.864880 (+ 179us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:55:20.865461 (+ 581us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 06:55:20.865467 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 06:55:20.865756 (+ 289us) client_negotiation.cc:769] Sending connection context
0812 06:55:20.865937 (+ 181us) client_negotiation.cc:240] Negotiation successful
0812 06:55:20.866237 (+ 300us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":276,"thread_start_us":118,"threads_started":1}
I20260812 06:55:20.868827 26732 heartbeater.cc:347] Connected to a master server at 127.19.52.126:40117
I20260812 06:55:20.869148 26732 heartbeater.cc:464] Registering TS with master...
I20260812 06:55:20.869781 26732 heartbeater.cc:511] Master 127.19.52.126:40117 requested a full tablet report, sending...
I20260812 06:55:20.870864 26186 ts_manager.cc:194] Registered new tserver with Master: 4f6fff7c528a4c41ad7dbf8b925715d0 (127.19.52.67:44377)
I20260812 06:55:20.871487 26186 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.52.67@KRBTEST.COM'} at 127.19.52.67:56701
I20260812 06:55:20.881891 19665 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal test-admin with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786517687384214-19665-0/krb5kdc/test-admin.keytab.
Entry for principal test-admin with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786517687384214-19665-0/krb5kdc/test-admin.keytab.
Aug 12 06:55:20 dist-test-slave-65mx krb5kdc[26141](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517720, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 06:55:20.903002 19665 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
W20260812 06:55:20.906555 26740 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 06:55:20.904674 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43824 (local address 127.19.52.65:36141)
0812 06:55:20.905290 (+ 616us) server_negotiation.cc:206] Beginning negotiation
0812 06:55:20.905296 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 06:55:20.905321 (+ 25us) server_negotiation.cc:407] Connection header received
0812 06:55:20.905372 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:55:20.905377 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:55:20.905567 (+ 190us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:55:20.905696 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:55:20.906377 (+ 681us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43824: BlockingRecv error: recv got EOF from 127.0.0.1:43824 (error 108)
Metrics: {"server-negotiator.queue_time_us":509,"thread_start_us":180,"threads_started":1}
W20260812 06:55:20.906916 19665 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786517687384214-19665-0/krb5kdc/krb5cc' of type FILE
Aug 12 06:55:20 dist-test-slave-65mx krb5kdc[26141](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517720, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 06:55:20.908166 19665 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 12 06:55:20 dist-test-slave-65mx krb5kdc[26141](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517720, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 06:55:20.909391 19665 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 12 06:55:20 dist-test-slave-65mx krb5kdc[26141](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517720, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.19.52.65@KRBTEST.COM
I20260812 06:55:20.921420 26740 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:20.910547 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43826 (local address 127.19.52.65:36141)
0812 06:55:20.910713 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0812 06:55:20.910720 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 06:55:20.910833 (+ 113us) server_negotiation.cc:407] Connection header received
0812 06:55:20.911080 (+ 247us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:55:20.911082 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:55:20.911132 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:55:20.911226 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:55:20.911854 (+ 628us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:20.912859 (+ 1005us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:20.913921 (+ 1062us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:20.914361 (+ 440us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:20.914503 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:55:20.917336 (+ 2833us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:55:20.917346 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:55:20.917360 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:55:20.917387 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:55:20.919301 (+ 1914us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:55:20.919800 (+ 499us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:55:20.919806 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:55:20.919807 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:55:20.919897 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:55:20.920207 (+ 310us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:55:20.920210 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:55:20.920212 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:55:20.920507 (+ 295us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:55:20.920826 (+ 319us) server_negotiation.cc:1035] Waiting for connection context
0812 06:55:20.921147 (+ 321us) server_negotiation.cc:299] Negotiation successful
0812 06:55:20.921285 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
W20260812 06:55:20.926131 26740 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 06:55:20.924766 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43834 (local address 127.19.52.65:36141)
0812 06:55:20.924918 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0812 06:55:20.924925 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 06:55:20.925131 (+ 206us) server_negotiation.cc:407] Connection header received
0812 06:55:20.925303 (+ 172us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:55:20.925307 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:55:20.925360 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:55:20.925456 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:55:20.925974 (+ 518us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43834: BlockingRecv error: recv got EOF from 127.0.0.1:43834 (error 108)
Metrics: {"server-negotiator.queue_time_us":48}
W20260812 06:55:20.926556 19665 init.cc:244] Error while opening credential cache '/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786517687384214-19665-0/krb5kdc/krb5cc' of type FILE: Runtime error: End of credential cache reached (filename: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786517687384214-19665-0/krb5kdc/krb5cc)
Aug 12 06:55:20 dist-test-slave-65mx krb5kdc[26141](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517720, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 06:55:20.927740 19665 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 12 06:55:20 dist-test-slave-65mx krb5kdc[26141](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517720, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 06:55:20.928844 19665 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 12 06:55:20 dist-test-slave-65mx krb5kdc[26141](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517720, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.19.52.65@KRBTEST.COM
I20260812 06:55:20.938496 26740 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:20.929826 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43846 (local address 127.19.52.65:36141)
0812 06:55:20.929961 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0812 06:55:20.929965 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 06:55:20.930241 (+ 276us) server_negotiation.cc:407] Connection header received
0812 06:55:20.930336 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:55:20.930339 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:55:20.930381 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:55:20.930455 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:55:20.931071 (+ 616us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:20.931574 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:20.932345 (+ 771us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:20.932697 (+ 352us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:20.932783 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:55:20.935275 (+ 2492us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:55:20.935279 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:55:20.935280 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:55:20.935307 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:55:20.936887 (+ 1580us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:55:20.937325 (+ 438us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:55:20.937327 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:55:20.937329 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:55:20.937405 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:55:20.937696 (+ 291us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:55:20.937698 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:55:20.937699 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:55:20.937874 (+ 175us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:55:20.938036 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0812 06:55:20.938179 (+ 143us) server_negotiation.cc:299] Negotiation successful
0812 06:55:20.938278 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
W20260812 06:55:20.941705 26740 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 06:55:20.940371 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43856 (local address 127.19.52.65:36141)
0812 06:55:20.940626 (+ 255us) server_negotiation.cc:206] Beginning negotiation
0812 06:55:20.940631 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 06:55:20.940700 (+ 69us) server_negotiation.cc:407] Connection header received
0812 06:55:20.940869 (+ 169us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:55:20.940872 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:55:20.940919 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:55:20.940983 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:55:20.941608 (+ 625us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43856: BlockingRecv error: recv got EOF from 127.0.0.1:43856 (error 108)
Metrics: {"server-negotiator.queue_time_us":110}
W20260812 06:55:20.942134 19665 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786517687384214-19665-0/krb5kdc/krb5cc' of type FILE
Aug 12 06:55:20 dist-test-slave-65mx krb5kdc[26141](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517720, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 06:55:20.943238 19665 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 12 06:55:20 dist-test-slave-65mx krb5kdc[26141](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517720, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 06:55:20.944164 19665 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 12 06:55:20 dist-test-slave-65mx krb5kdc[26141](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517720, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.19.52.65@KRBTEST.COM
I20260812 06:55:20.953650 26740 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:20.945063 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43862 (local address 127.19.52.65:36141)
0812 06:55:20.945216 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0812 06:55:20.945220 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 06:55:20.945401 (+ 181us) server_negotiation.cc:407] Connection header received
0812 06:55:20.945453 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:55:20.945455 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:55:20.945495 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:55:20.945569 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:55:20.946224 (+ 655us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:20.946744 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:20.947616 (+ 872us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:20.947847 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:20.947935 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:55:20.950342 (+ 2407us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:55:20.950345 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:55:20.950346 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:55:20.950368 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:55:20.951804 (+ 1436us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:55:20.952218 (+ 414us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:55:20.952221 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:55:20.952222 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:55:20.952299 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:55:20.952754 (+ 455us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:55:20.952757 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:55:20.952759 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:55:20.952965 (+ 206us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:55:20.953129 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0812 06:55:20.953375 (+ 246us) server_negotiation.cc:299] Negotiation successful
0812 06:55:20.953514 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260812 06:55:20.955127 19665 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu with pid 26264
I20260812 06:55:20.966089 19665 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu with pid 26422
I20260812 06:55:20.974072 19665 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu with pid 26583
I20260812 06:55:20.984072 19665 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu with pid 26157
2026-08-12T06:55:20Z chronyd exiting
[ OK ] SecurityITest.TestCorruptKerberosCC (2433 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipalMultipleMaster
Loading random data
Initializing database '/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517687384214-19665-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 06:55:21 dist-test-slave-65mx krb5kdc[26753](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 06:55:21 dist-test-slave-65mx krb5kdc[26753](info): set up 2 sockets
Aug 12 06:55:21 dist-test-slave-65mx krb5kdc[26753](info): commencing operation
krb5kdc: starting...
W20260812 06:55:22.138774 19665 mini_kdc.cc:121] Time spent starting KDC: real 1.129s user 0.003s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517687384214-19665-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517687384214-19665-0/krb5kdc/test-user.keytab.
Aug 12 06:55:22 dist-test-slave-65mx krb5kdc[26753](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517722, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T06:55:22Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T06:55:22Z Disabled control of system clock
WARNING: no policy specified for oryx/127.19.52.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.19.52.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.19.52.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517687384214-19665-0/krb5kdc/oryx_127.19.52.126.keytab.
Entry for principal oryx/127.19.52.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517687384214-19665-0/krb5kdc/oryx_127.19.52.126.keytab.
I20260812 06:55:22.289464 19665 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu
/tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517687384214-19665-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517687384214-19665-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517687384214-19665-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517687384214-19665-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.52.126:34933
--webserver_interface=127.19.52.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517687384214-19665-0/krb5kdc/oryx_127.19.52.126.keytab
--principal=oryx/127.19.52.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.52.84:43707
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.19.52.126:34933
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517687384214-19665-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517687384214-19665-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517687384214-19665-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 06:55:22.407399 26768 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:55:22.407748 26768 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:55:22.407840 26768 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 06:55:22.412117 26768 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 06:55:22.412240 26768 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 06:55:22.412281 26768 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 06:55:22.412318 26768 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 06:55:22.412351 26768 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 06:55:22.418466 26768 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.52.84:43707
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517687384214-19665-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517687384214-19665-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.19.52.126:34933
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517687384214-19665-0/krb5kdc/oryx_127.19.52.126.keytab
--principal=oryx/127.19.52.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.52.126:34933
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517687384214-19665-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.19.52.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26768
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517687384214-19665-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 06:15:54 UTC on ccycloud.quasar-rnfzce.root.comops.site
build id 4581989
I20260812 06:55:22.419674 26768 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:55:22.420802 26768 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260812 06:55:22.430629 26768 server_base.cc:1034] running on GCE node
W20260812 06:55:22.430503 26778 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:55:22.430503 26776 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:55:22.430523 26775 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:55:22.431509 26768 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:55:22.432893 26768 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:55:22.434125 26768 hybrid_clock.cc:648] HybridClock initialized: now 1786517722434110 us; error 31 us; skew 500 ppm
Aug 12 06:55:22 dist-test-slave-65mx krb5kdc[26753](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517722, etypes {rep=17 tkt=17 ses=17}, oryx/127.19.52.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 06:55:22.437729 26768 init.cc:377] Logged in from keytab as oryx/127.19.52.126@KRBTEST.COM (short username oryx)
I20260812 06:55:22.438832 26768 webserver.cc:460] Webserver started at http://127.19.52.126:36975/ using document root <none> and password file <none>
I20260812 06:55:22.439504 26768 fs_manager.cc:359] Metadata directory not provided
I20260812 06:55:22.439615 26768 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:55:22.439886 26768 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 06:55:22.442446 26768 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517687384214-19665-0/minicluster-data/master-0/data/instance:
uuid: "53e6ebc3b8ee4672983fb82b50148e26"
format_stamp: "Formatted at 2026-08-12 06:55:22 on dist-test-slave-65mx"
server_key: "8cd3e6d4b77a0c0fe79beff024214c76"
server_key_iv: "a3304faea48b562269147140fec0fe99"
server_key_version: "encryptionkey@0"
I20260812 06:55:22.443132 26768 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517687384214-19665-0/minicluster-data/master-0/wal/instance:
uuid: "53e6ebc3b8ee4672983fb82b50148e26"
format_stamp: "Formatted at 2026-08-12 06:55:22 on dist-test-slave-65mx"
server_key: "8cd3e6d4b77a0c0fe79beff024214c76"
server_key_iv: "a3304faea48b562269147140fec0fe99"
server_key_version: "encryptionkey@0"
I20260812 06:55:22.447574 26768 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.001s
I20260812 06:55:22.450496 26786 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 06:55:22.451928 26768 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260812 06:55:22.452113 26768 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517687384214-19665-0/minicluster-data/master-0/data,/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517687384214-19665-0/minicluster-data/master-0/wal
uuid: "53e6ebc3b8ee4672983fb82b50148e26"
format_stamp: "Formatted at 2026-08-12 06:55:22 on dist-test-slave-65mx"
server_key: "8cd3e6d4b77a0c0fe79beff024214c76"
server_key_iv: "a3304faea48b562269147140fec0fe99"
server_key_version: "encryptionkey@0"
I20260812 06:55:22.452251 26768 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517687384214-19665-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517687384214-19665-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517687384214-19665-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:55:22.493878 26768 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 06:55:22.497884 26768 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 06:55:22.498241 26768 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:55:22.506282 26768 rpc_server.cc:307] RPC server started. Bound to: 127.19.52.126:34933
I20260812 06:55:22.506606 26843 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.52.126:34933 every 8 connection(s)
I20260812 06:55:22.507436 26768 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517687384214-19665-0/minicluster-data/master-0/data/info.pb
I20260812 06:55:22.508798 19665 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu as pid 26768
I20260812 06:55:22.508953 19665 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517687384214-19665-0/minicluster-data/master-0/wal/instance
I20260812 06:55:22.509297 19665 external_mini_cluster.cc:1428] Setting key a6f9ccfe9d502625cdb1c5da0e0b665c
I20260812 06:55:22.511272 26845 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 06:55:22.517123 26845 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 53e6ebc3b8ee4672983fb82b50148e26: Bootstrap starting.
Aug 12 06:55:22 dist-test-slave-65mx krb5kdc[26753](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517722, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.19.52.126@KRBTEST.COM
I20260812 06:55:22.519815 26845 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 53e6ebc3b8ee4672983fb82b50148e26: Neither blocks nor log segments found. Creating new log.
I20260812 06:55:22.520788 26845 log.cc:824] T 00000000000000000000000000000000 P 53e6ebc3b8ee4672983fb82b50148e26: Log is configured to *not* fsync() on all Append() calls
I20260812 06:55:22.523140 26845 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 53e6ebc3b8ee4672983fb82b50148e26: No bootstrap required, opened a new log
I20260812 06:55:22.525081 26848 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:22.510767 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44446 (local address 127.19.52.126:34933)
0812 06:55:22.511252 (+ 485us) server_negotiation.cc:206] Beginning negotiation
0812 06:55:22.511268 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0812 06:55:22.511303 (+ 35us) server_negotiation.cc:407] Connection header received
0812 06:55:22.512027 (+ 724us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:55:22.512044 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:55:22.512372 (+ 328us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:55:22.512765 (+ 393us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:55:22.513589 (+ 824us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:22.514594 (+ 1005us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:22.515449 (+ 855us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:22.515934 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:22.516110 (+ 176us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:55:22.519223 (+ 3113us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:55:22.519229 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:55:22.519242 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:55:22.519272 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:55:22.521513 (+ 2241us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:55:22.522096 (+ 583us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:55:22.522101 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:55:22.522105 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:55:22.522220 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:55:22.522567 (+ 347us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:55:22.522570 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:55:22.522571 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:55:22.523061 (+ 490us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:55:22.523270 (+ 209us) server_negotiation.cc:1035] Waiting for connection context
0812 06:55:22.523609 (+ 339us) server_negotiation.cc:299] Negotiation successful
0812 06:55:22.523989 (+ 380us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":342,"thread_start_us":167,"threads_started":1}
I20260812 06:55:22.526422 26845 raft_consensus.cc:348] T 00000000000000000000000000000000 P 53e6ebc3b8ee4672983fb82b50148e26 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "53e6ebc3b8ee4672983fb82b50148e26" member_type: VOTER last_known_addr { host: "127.19.52.126" port: 34933 } }
I20260812 06:55:22.526641 26845 raft_consensus.cc:374] T 00000000000000000000000000000000 P 53e6ebc3b8ee4672983fb82b50148e26 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 06:55:22.526702 26845 raft_consensus.cc:729] T 00000000000000000000000000000000 P 53e6ebc3b8ee4672983fb82b50148e26 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 53e6ebc3b8ee4672983fb82b50148e26, State: Initialized, Role: FOLLOWER
I20260812 06:55:22.527109 26845 consensus_queue.cc:260] T 00000000000000000000000000000000 P 53e6ebc3b8ee4672983fb82b50148e26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "53e6ebc3b8ee4672983fb82b50148e26" member_type: VOTER last_known_addr { host: "127.19.52.126" port: 34933 } }
I20260812 06:55:22.527226 26845 raft_consensus.cc:388] T 00000000000000000000000000000000 P 53e6ebc3b8ee4672983fb82b50148e26 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 06:55:22.527308 26845 raft_consensus.cc:482] T 00000000000000000000000000000000 P 53e6ebc3b8ee4672983fb82b50148e26 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 06:55:22.527411 26845 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 53e6ebc3b8ee4672983fb82b50148e26 [term 0 FOLLOWER]: Advancing to term 1
I20260812 06:55:22.528419 26845 raft_consensus.cc:504] T 00000000000000000000000000000000 P 53e6ebc3b8ee4672983fb82b50148e26 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "53e6ebc3b8ee4672983fb82b50148e26" member_type: VOTER last_known_addr { host: "127.19.52.126" port: 34933 } }
I20260812 06:55:22.528759 26845 leader_election.cc:302] T 00000000000000000000000000000000 P 53e6ebc3b8ee4672983fb82b50148e26 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 53e6ebc3b8ee4672983fb82b50148e26; no voters:
I20260812 06:55:22.529031 26845 leader_election.cc:288] T 00000000000000000000000000000000 P 53e6ebc3b8ee4672983fb82b50148e26 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 06:55:22.529266 26850 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 53e6ebc3b8ee4672983fb82b50148e26 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 06:55:22.529614 26850 raft_consensus.cc:686] T 00000000000000000000000000000000 P 53e6ebc3b8ee4672983fb82b50148e26 [term 1 LEADER]: Becoming Leader. State: Replica: 53e6ebc3b8ee4672983fb82b50148e26, State: Running, Role: LEADER
I20260812 06:55:22.529974 26850 consensus_queue.cc:237] T 00000000000000000000000000000000 P 53e6ebc3b8ee4672983fb82b50148e26 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "53e6ebc3b8ee4672983fb82b50148e26" member_type: VOTER last_known_addr { host: "127.19.52.126" port: 34933 } }
I20260812 06:55:22.530273 26845 sys_catalog.cc:564] T 00000000000000000000000000000000 P 53e6ebc3b8ee4672983fb82b50148e26 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 06:55:22.531698 26853 sys_catalog.cc:455] T 00000000000000000000000000000000 P 53e6ebc3b8ee4672983fb82b50148e26 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 53e6ebc3b8ee4672983fb82b50148e26. Latest consensus state: current_term: 1 leader_uuid: "53e6ebc3b8ee4672983fb82b50148e26" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "53e6ebc3b8ee4672983fb82b50148e26" member_type: VOTER last_known_addr { host: "127.19.52.126" port: 34933 } } }
I20260812 06:55:22.531850 26853 sys_catalog.cc:458] T 00000000000000000000000000000000 P 53e6ebc3b8ee4672983fb82b50148e26 [sys.catalog]: This master's current role is: LEADER
I20260812 06:55:22.532245 26851 sys_catalog.cc:455] T 00000000000000000000000000000000 P 53e6ebc3b8ee4672983fb82b50148e26 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "53e6ebc3b8ee4672983fb82b50148e26" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "53e6ebc3b8ee4672983fb82b50148e26" member_type: VOTER last_known_addr { host: "127.19.52.126" port: 34933 } } }
I20260812 06:55:22.532335 26851 sys_catalog.cc:458] T 00000000000000000000000000000000 P 53e6ebc3b8ee4672983fb82b50148e26 [sys.catalog]: This master's current role is: LEADER
W20260812 06:55:22.535187 26869 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 53e6ebc3b8ee4672983fb82b50148e26: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 06:55:22.535270 26869 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 06:55:22.535451 26866 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 06:55:22.536276 26866 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 06:55:22.542770 26866 catalog_manager.cc:1347] Generated new cluster ID: 020dffafdb2c4d38b19668de7f5df860
I20260812 06:55:22.542852 26866 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 06:55:22.550132 26866 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 06:55:22.551250 26866 catalog_manager.cc:1504] Loading token signing keys...
I20260812 06:55:22.562002 26866 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 53e6ebc3b8ee4672983fb82b50148e26: Generated new TSK 0
I20260812 06:55:22.563220 26866 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.19.52.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.19.52.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.19.52.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517687384214-19665-0/krb5kdc/oryx_127.19.52.65.keytab.
Entry for principal oryx/127.19.52.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517687384214-19665-0/krb5kdc/oryx_127.19.52.65.keytab.
I20260812 06:55:22.600924 19665 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu
/tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517687384214-19665-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517687384214-19665-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517687384214-19665-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517687384214-19665-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.52.65:0
--local_ip_for_outbound_sockets=127.19.52.65
--webserver_interface=127.19.52.65
--webserver_port=0
--tserver_master_addrs=127.19.52.126:34933
--keytab_file=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517687384214-19665-0/krb5kdc/oryx_127.19.52.65.keytab
--principal=oryx/127.19.52.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.52.84:43707
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517687384214-19665-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517687384214-19665-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517687384214-19665-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 06:55:22.733213 26875 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:55:22.733516 26875 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:55:22.733595 26875 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 06:55:22.737807 26875 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 06:55:22.737922 26875 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 06:55:22.738052 26875 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.52.65
I20260812 06:55:22.743713 26875 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.52.84:43707
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517687384214-19665-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517687384214-19665-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517687384214-19665-0/krb5kdc/oryx_127.19.52.65.keytab
--principal=oryx/127.19.52.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.52.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517687384214-19665-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.19.52.65
--webserver_port=0
--tserver_master_addrs=127.19.52.126:34933
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26875
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.52.65
--log_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517687384214-19665-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 06:15:54 UTC on ccycloud.quasar-rnfzce.root.comops.site
build id 4581989
I20260812 06:55:22.744913 26875 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:55:22.745939 26875 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 06:55:22.756734 26880 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:55:22.756829 26875 server_base.cc:1034] running on GCE node
W20260812 06:55:22.756734 26883 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:55:22.756758 26881 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:55:22.757555 26875 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:55:22.758364 26875 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:55:22.759572 26875 hybrid_clock.cc:648] HybridClock initialized: now 1786517722759555 us; error 31 us; skew 500 ppm
Aug 12 06:55:22 dist-test-slave-65mx krb5kdc[26753](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517722, etypes {rep=17 tkt=17 ses=17}, oryx/127.19.52.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 06:55:22.762760 26875 init.cc:377] Logged in from keytab as oryx/127.19.52.65@KRBTEST.COM (short username oryx)
I20260812 06:55:22.764055 26875 webserver.cc:460] Webserver started at http://127.19.52.65:39559/ using document root <none> and password file <none>
I20260812 06:55:22.764945 26875 fs_manager.cc:359] Metadata directory not provided
I20260812 06:55:22.765079 26875 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:55:22.765358 26875 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 06:55:22.767732 26875 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517687384214-19665-0/minicluster-data/ts-0/data/instance:
uuid: "49e259bbf4094665b4e899332664889c"
format_stamp: "Formatted at 2026-08-12 06:55:22 on dist-test-slave-65mx"
server_key: "7c90dd6a5388828c4ca99ec509ea42f5"
server_key_iv: "da23e9e61d2d1349b6ecc96ac85fa6a1"
server_key_version: "encryptionkey@0"
I20260812 06:55:22.768405 26875 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517687384214-19665-0/minicluster-data/ts-0/wal/instance:
uuid: "49e259bbf4094665b4e899332664889c"
format_stamp: "Formatted at 2026-08-12 06:55:22 on dist-test-slave-65mx"
server_key: "7c90dd6a5388828c4ca99ec509ea42f5"
server_key_iv: "da23e9e61d2d1349b6ecc96ac85fa6a1"
server_key_version: "encryptionkey@0"
I20260812 06:55:22.772661 26875 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260812 06:55:22.775388 26891 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 06:55:22.776757 26875 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 06:55:22.776898 26875 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517687384214-19665-0/minicluster-data/ts-0/data,/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517687384214-19665-0/minicluster-data/ts-0/wal
uuid: "49e259bbf4094665b4e899332664889c"
format_stamp: "Formatted at 2026-08-12 06:55:22 on dist-test-slave-65mx"
server_key: "7c90dd6a5388828c4ca99ec509ea42f5"
server_key_iv: "da23e9e61d2d1349b6ecc96ac85fa6a1"
server_key_version: "encryptionkey@0"
I20260812 06:55:22.777051 26875 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517687384214-19665-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517687384214-19665-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517687384214-19665-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:55:22.801450 26875 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 06:55:22.804816 26875 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 06:55:22.805065 26875 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:55:22.805728 26875 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 06:55:22.806629 26875 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 06:55:22.806705 26875 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 06:55:22.806782 26875 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 06:55:22.806828 26875 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 06:55:22.816464 26875 rpc_server.cc:307] RPC server started. Bound to: 127.19.52.65:41417
I20260812 06:55:22.816525 27019 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.52.65:41417 every 8 connection(s)
I20260812 06:55:22.817687 26875 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517687384214-19665-0/minicluster-data/ts-0/data/info.pb
I20260812 06:55:22.820225 19665 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu as pid 26875
I20260812 06:55:22.820358 19665 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517687384214-19665-0/minicluster-data/ts-0/wal/instance
I20260812 06:55:22.820737 19665 external_mini_cluster.cc:1428] Setting key 56baf74079a2a8a66683b4ef23c068df
Aug 12 06:55:22 dist-test-slave-65mx krb5kdc[26753](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517722, etypes {rep=17 tkt=17 ses=17}, oryx/127.19.52.65@KRBTEST.COM for oryx/127.19.52.126@KRBTEST.COM
WARNING: no policy specified for oryx/127.19.52.66@KRBTEST.COM; defaulting to no policy
I20260812 06:55:22.834442 26848 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:22.819732 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.52.65:55221 (local address 127.19.52.126:34933)
0812 06:55:22.819911 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0812 06:55:22.819916 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 06:55:22.820905 (+ 989us) server_negotiation.cc:407] Connection header received
0812 06:55:22.822747 (+ 1842us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:55:22.822751 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:55:22.822807 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:55:22.822914 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:55:22.824137 (+ 1223us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:22.824974 (+ 837us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:22.826213 (+ 1239us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:22.826637 (+ 424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:22.826787 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:55:22.829901 (+ 3114us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:55:22.829906 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:55:22.829911 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:55:22.829949 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:55:22.831942 (+ 1993us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:55:22.832692 (+ 750us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:55:22.832698 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:55:22.832703 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:55:22.832804 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:55:22.833246 (+ 442us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:55:22.833249 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:55:22.833250 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:55:22.833531 (+ 281us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:55:22.833623 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0812 06:55:22.834184 (+ 561us) server_negotiation.cc:299] Negotiation successful
0812 06:55:22.834311 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260812 06:55:22.835593 27022 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:22.820127 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.126:34933 (local address 127.19.52.65:55221)
0812 06:55:22.820737 (+ 610us) negotiation.cc:107] Waiting for socket to connect
0812 06:55:22.820801 (+ 64us) client_negotiation.cc:174] Beginning negotiation
0812 06:55:22.822523 (+ 1722us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:55:22.823109 (+ 586us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:55:22.823116 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:55:22.823489 (+ 373us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:55:22.823975 (+ 486us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:55:22.823987 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:55:22.825133 (+ 1146us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:55:22.825136 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:55:22.826084 (+ 948us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:55:22.826094 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:55:22.826841 (+ 747us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:55:22.826845 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:55:22.827008 (+ 163us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:55:22.827772 (+ 764us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 06:55:22.827802 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:55:22.829688 (+ 1886us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 06:55:22.832098 (+ 2410us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:55:22.832104 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:55:22.832118 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:55:22.832587 (+ 469us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:55:22.832922 (+ 335us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:55:22.832926 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:55:22.832928 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:55:22.833099 (+ 171us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:55:22.833665 (+ 566us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 06:55:22.833670 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 06:55:22.833933 (+ 263us) client_negotiation.cc:769] Sending connection context
0812 06:55:22.834117 (+ 184us) client_negotiation.cc:240] Negotiation successful
0812 06:55:22.834402 (+ 285us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":351,"thread_start_us":164,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.19.52.66@KRBTEST.COM" created.
I20260812 06:55:22.837199 27020 heartbeater.cc:347] Connected to a master server at 127.19.52.126:34933
I20260812 06:55:22.837466 27020 heartbeater.cc:464] Registering TS with master...
I20260812 06:55:22.838016 27020 heartbeater.cc:511] Master 127.19.52.126:34933 requested a full tablet report, sending...
I20260812 06:55:22.839820 26803 ts_manager.cc:194] Registered new tserver with Master: 49e259bbf4094665b4e899332664889c (127.19.52.65:41417)
I20260812 06:55:22.841032 26803 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.19.52.65@KRBTEST.COM'} at 127.19.52.65:55221
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.19.52.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517687384214-19665-0/krb5kdc/oryx_127.19.52.66.keytab.
Entry for principal oryx/127.19.52.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517687384214-19665-0/krb5kdc/oryx_127.19.52.66.keytab.
I20260812 06:55:22.858615 19665 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu
/tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517687384214-19665-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517687384214-19665-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517687384214-19665-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517687384214-19665-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.52.66:0
--local_ip_for_outbound_sockets=127.19.52.66
--webserver_interface=127.19.52.66
--webserver_port=0
--tserver_master_addrs=127.19.52.126:34933
--keytab_file=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517687384214-19665-0/krb5kdc/oryx_127.19.52.66.keytab
--principal=oryx/127.19.52.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.52.84:43707
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517687384214-19665-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517687384214-19665-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517687384214-19665-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 06:55:22.977676 27025 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:55:22.978039 27025 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:55:22.978123 27025 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 06:55:22.982575 27025 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 06:55:22.982679 27025 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 06:55:22.982776 27025 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.52.66
I20260812 06:55:22.988385 27025 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.52.84:43707
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517687384214-19665-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517687384214-19665-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517687384214-19665-0/krb5kdc/oryx_127.19.52.66.keytab
--principal=oryx/127.19.52.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.52.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517687384214-19665-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.19.52.66
--webserver_port=0
--tserver_master_addrs=127.19.52.126:34933
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27025
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.52.66
--log_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517687384214-19665-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 06:15:54 UTC on ccycloud.quasar-rnfzce.root.comops.site
build id 4581989
I20260812 06:55:22.989610 27025 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:55:22.990669 27025 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 06:55:23.000851 27031 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:55:23.000900 27033 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:55:23.000960 27035 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:55:23.001132 27025 server_base.cc:1034] running on GCE node
I20260812 06:55:23.001668 27025 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:55:23.002362 27025 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:55:23.003577 27025 hybrid_clock.cc:648] HybridClock initialized: now 1786517723003557 us; error 31 us; skew 500 ppm
Aug 12 06:55:23 dist-test-slave-65mx krb5kdc[26753](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517723, etypes {rep=17 tkt=17 ses=17}, oryx/127.19.52.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 06:55:23.006744 27025 init.cc:377] Logged in from keytab as oryx/127.19.52.66@KRBTEST.COM (short username oryx)
I20260812 06:55:23.007707 27025 webserver.cc:460] Webserver started at http://127.19.52.66:35657/ using document root <none> and password file <none>
I20260812 06:55:23.008308 27025 fs_manager.cc:359] Metadata directory not provided
I20260812 06:55:23.008375 27025 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:55:23.008711 27025 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 06:55:23.011045 27025 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517687384214-19665-0/minicluster-data/ts-1/data/instance:
uuid: "2e97338a875f4bada5bf4e6301558f78"
format_stamp: "Formatted at 2026-08-12 06:55:23 on dist-test-slave-65mx"
server_key: "8725a83480edc9ec5c00367522d80e77"
server_key_iv: "ecde9130367d813eeaefa9f7eaef66fc"
server_key_version: "encryptionkey@0"
I20260812 06:55:23.011706 27025 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517687384214-19665-0/minicluster-data/ts-1/wal/instance:
uuid: "2e97338a875f4bada5bf4e6301558f78"
format_stamp: "Formatted at 2026-08-12 06:55:23 on dist-test-slave-65mx"
server_key: "8725a83480edc9ec5c00367522d80e77"
server_key_iv: "ecde9130367d813eeaefa9f7eaef66fc"
server_key_version: "encryptionkey@0"
I20260812 06:55:23.016456 27025 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.007s sys 0.000s
I20260812 06:55:23.019280 27043 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 06:55:23.020816 27025 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.003s
I20260812 06:55:23.020982 27025 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517687384214-19665-0/minicluster-data/ts-1/data,/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517687384214-19665-0/minicluster-data/ts-1/wal
uuid: "2e97338a875f4bada5bf4e6301558f78"
format_stamp: "Formatted at 2026-08-12 06:55:23 on dist-test-slave-65mx"
server_key: "8725a83480edc9ec5c00367522d80e77"
server_key_iv: "ecde9130367d813eeaefa9f7eaef66fc"
server_key_version: "encryptionkey@0"
I20260812 06:55:23.021103 27025 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517687384214-19665-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517687384214-19665-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517687384214-19665-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:55:23.037214 27025 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 06:55:23.040885 27025 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 06:55:23.041116 27025 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:55:23.041911 27025 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 06:55:23.042838 27025 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 06:55:23.042891 27025 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 06:55:23.042961 27025 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 06:55:23.042987 27025 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 06:55:23.052919 27025 rpc_server.cc:307] RPC server started. Bound to: 127.19.52.66:34045
I20260812 06:55:23.052945 27176 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.52.66:34045 every 8 connection(s)
I20260812 06:55:23.054050 27025 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517687384214-19665-0/minicluster-data/ts-1/data/info.pb
I20260812 06:55:23.057659 19665 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu as pid 27025
I20260812 06:55:23.057796 19665 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517687384214-19665-0/minicluster-data/ts-1/wal/instance
I20260812 06:55:23.058174 19665 external_mini_cluster.cc:1428] Setting key ad0f821eaac7e3c6762a1c5f08f2245d
Aug 12 06:55:23 dist-test-slave-65mx krb5kdc[26753](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517723, etypes {rep=17 tkt=17 ses=17}, oryx/127.19.52.66@KRBTEST.COM for oryx/127.19.52.126@KRBTEST.COM
WARNING: no policy specified for oryx/127.19.52.67@KRBTEST.COM; defaulting to no policy
I20260812 06:55:23.071544 26848 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:23.056231 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.52.66:53955 (local address 127.19.52.126:34933)
0812 06:55:23.056382 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0812 06:55:23.056386 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 06:55:23.057737 (+ 1351us) server_negotiation.cc:407] Connection header received
0812 06:55:23.059492 (+ 1755us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:55:23.059496 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:55:23.059555 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:55:23.059668 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:55:23.061427 (+ 1759us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:23.062036 (+ 609us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:23.063419 (+ 1383us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:23.063783 (+ 364us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:23.063876 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:55:23.067015 (+ 3139us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:55:23.067022 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:55:23.067029 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:55:23.067072 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:55:23.069022 (+ 1950us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:55:23.069732 (+ 710us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:55:23.069739 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:55:23.069744 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:55:23.069846 (+ 102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:55:23.070249 (+ 403us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:55:23.070254 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:55:23.070256 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:55:23.070551 (+ 295us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:55:23.070676 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0812 06:55:23.071244 (+ 568us) server_negotiation.cc:299] Negotiation successful
0812 06:55:23.071369 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260812 06:55:23.072432 27180 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:23.056841 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.126:34933 (local address 127.19.52.66:53955)
0812 06:55:23.057491 (+ 650us) negotiation.cc:107] Waiting for socket to connect
0812 06:55:23.057581 (+ 90us) client_negotiation.cc:174] Beginning negotiation
0812 06:55:23.059242 (+ 1661us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:55:23.059863 (+ 621us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:55:23.059871 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:55:23.060406 (+ 535us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:55:23.061231 (+ 825us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:55:23.061259 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:55:23.062207 (+ 948us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:55:23.062213 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:55:23.063279 (+ 1066us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:55:23.063289 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:55:23.063947 (+ 658us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:55:23.063951 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:55:23.064232 (+ 281us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:55:23.065089 (+ 857us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 06:55:23.065108 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:55:23.066771 (+ 1663us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 06:55:23.069200 (+ 2429us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:55:23.069207 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:55:23.069223 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:55:23.069583 (+ 360us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:55:23.069963 (+ 380us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:55:23.069965 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:55:23.069967 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:55:23.070131 (+ 164us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:55:23.070690 (+ 559us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 06:55:23.070697 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 06:55:23.070974 (+ 277us) client_negotiation.cc:769] Sending connection context
0812 06:55:23.071154 (+ 180us) client_negotiation.cc:240] Negotiation successful
0812 06:55:23.071458 (+ 304us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":374,"thread_start_us":137,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.19.52.67@KRBTEST.COM" created.
I20260812 06:55:23.074076 27178 heartbeater.cc:347] Connected to a master server at 127.19.52.126:34933
I20260812 06:55:23.074373 27178 heartbeater.cc:464] Registering TS with master...
I20260812 06:55:23.075045 27178 heartbeater.cc:511] Master 127.19.52.126:34933 requested a full tablet report, sending...
I20260812 06:55:23.076390 26803 ts_manager.cc:194] Registered new tserver with Master: 2e97338a875f4bada5bf4e6301558f78 (127.19.52.66:34045)
I20260812 06:55:23.077045 26803 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.19.52.66@KRBTEST.COM'} at 127.19.52.66:53955
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.19.52.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517687384214-19665-0/krb5kdc/oryx_127.19.52.67.keytab.
Entry for principal oryx/127.19.52.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517687384214-19665-0/krb5kdc/oryx_127.19.52.67.keytab.
I20260812 06:55:23.096298 19665 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu
/tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517687384214-19665-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517687384214-19665-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517687384214-19665-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517687384214-19665-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.52.67:0
--local_ip_for_outbound_sockets=127.19.52.67
--webserver_interface=127.19.52.67
--webserver_port=0
--tserver_master_addrs=127.19.52.126:34933
--keytab_file=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517687384214-19665-0/krb5kdc/oryx_127.19.52.67.keytab
--principal=oryx/127.19.52.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.52.84:43707
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517687384214-19665-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517687384214-19665-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517687384214-19665-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 06:55:23.215389 27183 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:55:23.215718 27183 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:55:23.215806 27183 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 06:55:23.220139 27183 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 06:55:23.220244 27183 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 06:55:23.220336 27183 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.52.67
I20260812 06:55:23.226068 27183 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.52.84:43707
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517687384214-19665-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517687384214-19665-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517687384214-19665-0/krb5kdc/oryx_127.19.52.67.keytab
--principal=oryx/127.19.52.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.52.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517687384214-19665-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.19.52.67
--webserver_port=0
--tserver_master_addrs=127.19.52.126:34933
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27183
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.52.67
--log_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517687384214-19665-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 06:15:54 UTC on ccycloud.quasar-rnfzce.root.comops.site
build id 4581989
I20260812 06:55:23.227246 27183 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:55:23.228302 27183 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 06:55:23.238942 27192 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:55:23.238986 27189 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:55:23.238946 27188 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:55:23.239219 27183 server_base.cc:1034] running on GCE node
I20260812 06:55:23.239769 27183 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:55:23.240607 27183 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:55:23.241842 27183 hybrid_clock.cc:648] HybridClock initialized: now 1786517723241825 us; error 33 us; skew 500 ppm
Aug 12 06:55:23 dist-test-slave-65mx krb5kdc[26753](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517723, etypes {rep=17 tkt=17 ses=17}, oryx/127.19.52.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 06:55:23.244978 27183 init.cc:377] Logged in from keytab as oryx/127.19.52.67@KRBTEST.COM (short username oryx)
I20260812 06:55:23.245893 27183 webserver.cc:460] Webserver started at http://127.19.52.67:38493/ using document root <none> and password file <none>
I20260812 06:55:23.246436 27183 fs_manager.cc:359] Metadata directory not provided
I20260812 06:55:23.246497 27183 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:55:23.246676 27183 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 06:55:23.248845 27183 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517687384214-19665-0/minicluster-data/ts-2/data/instance:
uuid: "fff2f0f0327b47368c203f7f1aab31fd"
format_stamp: "Formatted at 2026-08-12 06:55:23 on dist-test-slave-65mx"
server_key: "882f8e9953ea68aefeb9ea6311bf1b6a"
server_key_iv: "3242c8c2026901056bbe3e383384a576"
server_key_version: "encryptionkey@0"
I20260812 06:55:23.249387 27183 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517687384214-19665-0/minicluster-data/ts-2/wal/instance:
uuid: "fff2f0f0327b47368c203f7f1aab31fd"
format_stamp: "Formatted at 2026-08-12 06:55:23 on dist-test-slave-65mx"
server_key: "882f8e9953ea68aefeb9ea6311bf1b6a"
server_key_iv: "3242c8c2026901056bbe3e383384a576"
server_key_version: "encryptionkey@0"
I20260812 06:55:23.253484 27183 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260812 06:55:23.256187 27201 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 06:55:23.257898 27183 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260812 06:55:23.258055 27183 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517687384214-19665-0/minicluster-data/ts-2/data,/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517687384214-19665-0/minicluster-data/ts-2/wal
uuid: "fff2f0f0327b47368c203f7f1aab31fd"
format_stamp: "Formatted at 2026-08-12 06:55:23 on dist-test-slave-65mx"
server_key: "882f8e9953ea68aefeb9ea6311bf1b6a"
server_key_iv: "3242c8c2026901056bbe3e383384a576"
server_key_version: "encryptionkey@0"
I20260812 06:55:23.258234 27183 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517687384214-19665-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517687384214-19665-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517687384214-19665-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:55:23.283646 27183 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 06:55:23.287356 27183 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 06:55:23.287600 27183 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:55:23.288251 27183 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 06:55:23.289248 27183 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 06:55:23.289327 27183 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 06:55:23.289402 27183 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 06:55:23.289455 27183 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 06:55:23.299578 27183 rpc_server.cc:307] RPC server started. Bound to: 127.19.52.67:45705
I20260812 06:55:23.299607 27332 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.52.67:45705 every 8 connection(s)
I20260812 06:55:23.300518 27183 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517687384214-19665-0/minicluster-data/ts-2/data/info.pb
I20260812 06:55:23.305159 19665 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu as pid 27183
I20260812 06:55:23.305298 19665 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517687384214-19665-0/minicluster-data/ts-2/wal/instance
I20260812 06:55:23.305603 19665 external_mini_cluster.cc:1428] Setting key a205a4b379c04284d493c0493b953140
Aug 12 06:55:23 dist-test-slave-65mx krb5kdc[26753](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517723, etypes {rep=17 tkt=17 ses=17}, oryx/127.19.52.67@KRBTEST.COM for oryx/127.19.52.126@KRBTEST.COM
I20260812 06:55:23.315671 26848 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:23.302485 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.52.67:49825 (local address 127.19.52.126:34933)
0812 06:55:23.302688 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0812 06:55:23.302692 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 06:55:23.303585 (+ 893us) server_negotiation.cc:407] Connection header received
0812 06:55:23.305118 (+ 1533us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:55:23.305122 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:55:23.305204 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:55:23.305285 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:55:23.306910 (+ 1625us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:23.307525 (+ 615us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:23.308942 (+ 1417us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:23.309189 (+ 247us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:23.309273 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:55:23.312023 (+ 2750us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:55:23.312026 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:55:23.312028 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:55:23.312053 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:55:23.313623 (+ 1570us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:55:23.314167 (+ 544us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:55:23.314171 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:55:23.314172 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:55:23.314250 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:55:23.314595 (+ 345us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:55:23.314597 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:55:23.314598 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:55:23.314798 (+ 200us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:55:23.314877 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0812 06:55:23.315430 (+ 553us) server_negotiation.cc:299] Negotiation successful
0812 06:55:23.315537 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":85}
I20260812 06:55:23.316668 27336 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:23.302897 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.126:34933 (local address 127.19.52.67:49825)
0812 06:55:23.303413 (+ 516us) negotiation.cc:107] Waiting for socket to connect
0812 06:55:23.303460 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0812 06:55:23.304878 (+ 1418us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:55:23.305540 (+ 662us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:55:23.305547 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:55:23.306079 (+ 532us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:55:23.306734 (+ 655us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:55:23.306753 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:55:23.307683 (+ 930us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:55:23.307686 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:55:23.308812 (+ 1126us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:55:23.308822 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:55:23.309342 (+ 520us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:55:23.309353 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:55:23.309562 (+ 209us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:55:23.310165 (+ 603us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 06:55:23.310192 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:55:23.311838 (+ 1646us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 06:55:23.313742 (+ 1904us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:55:23.313747 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:55:23.313760 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:55:23.314068 (+ 308us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:55:23.314337 (+ 269us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:55:23.314339 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:55:23.314341 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:55:23.314510 (+ 169us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:55:23.314915 (+ 405us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 06:55:23.314928 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 06:55:23.315192 (+ 264us) client_negotiation.cc:769] Sending connection context
0812 06:55:23.315363 (+ 171us) client_negotiation.cc:240] Negotiation successful
0812 06:55:23.315678 (+ 315us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":313,"thread_start_us":117,"threads_started":1}
I20260812 06:55:23.317955 27333 heartbeater.cc:347] Connected to a master server at 127.19.52.126:34933
I20260812 06:55:23.318241 27333 heartbeater.cc:464] Registering TS with master...
I20260812 06:55:23.318881 27333 heartbeater.cc:511] Master 127.19.52.126:34933 requested a full tablet report, sending...
I20260812 06:55:23.319902 26802 ts_manager.cc:194] Registered new tserver with Master: fff2f0f0327b47368c203f7f1aab31fd (127.19.52.67:45705)
I20260812 06:55:23.320533 26802 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.19.52.67@KRBTEST.COM'} at 127.19.52.67:49825
I20260812 06:55:23.332422 19665 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.19.52.125@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.19.52.125@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.19.52.125 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517687384214-19665-0/krb5kdc/oryx_127.19.52.125.keytab.
Entry for principal oryx/127.19.52.125 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517687384214-19665-0/krb5kdc/oryx_127.19.52.125.keytab.
I20260812 06:55:23.365196 19665 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu
/tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517687384214-19665-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517687384214-19665-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517687384214-19665-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517687384214-19665-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.52.125:32823
--webserver_interface=127.19.52.125
--webserver_port=0
--keytab_file=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517687384214-19665-0/krb5kdc/oryx_127.19.52.125.keytab
--principal=oryx/127.19.52.125
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.52.84:43707
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.19.52.126:34933,127.19.52.125:32823
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517687384214-19665-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517687384214-19665-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517687384214-19665-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 06:55:23.489295 27340 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:55:23.489631 27340 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:55:23.489675 27340 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 06:55:23.493973 27340 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 06:55:23.494056 27340 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 06:55:23.494076 27340 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 06:55:23.494094 27340 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 06:55:23.494109 27340 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 06:55:23.499979 27340 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.52.84:43707
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517687384214-19665-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517687384214-19665-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.19.52.126:34933,127.19.52.125:32823
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517687384214-19665-0/krb5kdc/oryx_127.19.52.125.keytab
--principal=oryx/127.19.52.125
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.52.125:32823
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517687384214-19665-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.19.52.125
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27340
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517687384214-19665-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 06:15:54 UTC on ccycloud.quasar-rnfzce.root.comops.site
build id 4581989
W20260812 06:55:23.500280 27340 master_options.cc:55] Only 2 masters are specified by master_addresses_flag ('127.19.52.126:34933,127.19.52.125:32823'), but minimum of 3 are required to tolerate failures of any one master. It is recommended to use at least 3 masters.
I20260812 06:55:23.501293 27340 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:55:23.502388 27340 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 06:55:23.511380 27345 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:55:23.511497 27346 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:55:23.511525 27340 server_base.cc:1034] running on GCE node
W20260812 06:55:23.511427 27348 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:55:23.512250 27340 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:55:23.513636 27340 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:55:23.514851 27340 hybrid_clock.cc:648] HybridClock initialized: now 1786517723514829 us; error 49 us; skew 500 ppm
Aug 12 06:55:23 dist-test-slave-65mx krb5kdc[26753](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517723, etypes {rep=17 tkt=17 ses=17}, oryx/127.19.52.125@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 06:55:23.517982 27340 init.cc:377] Logged in from keytab as oryx/127.19.52.125@KRBTEST.COM (short username oryx)
I20260812 06:55:23.519060 27340 webserver.cc:460] Webserver started at http://127.19.52.125:39537/ using document root <none> and password file <none>
I20260812 06:55:23.519747 27340 fs_manager.cc:359] Metadata directory not provided
I20260812 06:55:23.519816 27340 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:55:23.520048 27340 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 06:55:23.522224 27340 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517687384214-19665-0/minicluster-data/master-1/data/instance:
uuid: "2b5a510eb7be4cfab2f7914ebabaa929"
format_stamp: "Formatted at 2026-08-12 06:55:23 on dist-test-slave-65mx"
server_key: "63146d8807a10f93ad7c727bafd0367c"
server_key_iv: "27b0bd8d4674412ac6353b77d07bb687"
server_key_version: "encryptionkey@0"
I20260812 06:55:23.522846 27340 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517687384214-19665-0/minicluster-data/master-1/wal/instance:
uuid: "2b5a510eb7be4cfab2f7914ebabaa929"
format_stamp: "Formatted at 2026-08-12 06:55:23 on dist-test-slave-65mx"
server_key: "63146d8807a10f93ad7c727bafd0367c"
server_key_iv: "27b0bd8d4674412ac6353b77d07bb687"
server_key_version: "encryptionkey@0"
I20260812 06:55:23.526969 27340 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260812 06:55:23.529747 27357 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 06:55:23.531386 27340 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 06:55:23.531620 27340 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517687384214-19665-0/minicluster-data/master-1/data,/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517687384214-19665-0/minicluster-data/master-1/wal
uuid: "2b5a510eb7be4cfab2f7914ebabaa929"
format_stamp: "Formatted at 2026-08-12 06:55:23 on dist-test-slave-65mx"
server_key: "63146d8807a10f93ad7c727bafd0367c"
server_key_iv: "27b0bd8d4674412ac6353b77d07bb687"
server_key_version: "encryptionkey@0"
I20260812 06:55:23.531744 27340 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517687384214-19665-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517687384214-19665-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517687384214-19665-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:55:23.566990 27340 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 06:55:23.571008 27340 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 06:55:23.571269 27340 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:55:23.580037 27340 rpc_server.cc:307] RPC server started. Bound to: 127.19.52.125:32823
I20260812 06:55:23.580440 27418 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.52.125:32823 every 8 connection(s)
I20260812 06:55:23.581278 27340 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517687384214-19665-0/minicluster-data/master-1/data/info.pb
I20260812 06:55:23.584160 19665 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu as pid 27340
I20260812 06:55:23.584306 27419 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 06:55:23.589501 27419 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.19.52.126" port: 34933 } has no permanent_uuid. Determining permanent_uuid...
Aug 12 06:55:23 dist-test-slave-65mx krb5kdc[26753](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517722, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.19.52.125@KRBTEST.COM
Aug 12 06:55:23 dist-test-slave-65mx krb5kdc[26753](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517723, etypes {rep=17 tkt=17 ses=17}, oryx/127.19.52.125@KRBTEST.COM for oryx/127.19.52.126@KRBTEST.COM
I20260812 06:55:23.601497 27422 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:23.585765 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55208 (local address 127.19.52.125:32823)
0812 06:55:23.586191 (+ 426us) server_negotiation.cc:206] Beginning negotiation
0812 06:55:23.586207 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0812 06:55:23.586245 (+ 38us) server_negotiation.cc:407] Connection header received
0812 06:55:23.587170 (+ 925us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:55:23.587190 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:55:23.587542 (+ 352us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:55:23.587848 (+ 306us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:55:23.588865 (+ 1017us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:23.589987 (+ 1122us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:23.591152 (+ 1165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:23.591738 (+ 586us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:23.591902 (+ 164us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:55:23.594975 (+ 3073us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:55:23.594981 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:55:23.594997 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:55:23.595035 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:55:23.597543 (+ 2508us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:55:23.598140 (+ 597us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:55:23.598147 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:55:23.598154 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:55:23.598286 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:55:23.598708 (+ 422us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:55:23.598712 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:55:23.598714 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:55:23.599411 (+ 697us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:55:23.599675 (+ 264us) server_negotiation.cc:1035] Waiting for connection context
0812 06:55:23.599956 (+ 281us) server_negotiation.cc:299] Negotiation successful
0812 06:55:23.600383 (+ 427us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":278,"thread_start_us":156,"threads_started":1}
I20260812 06:55:23.602201 26848 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:23.591633 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44452 (local address 127.19.52.126:34933)
0812 06:55:23.591805 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0812 06:55:23.591812 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 06:55:23.592152 (+ 340us) server_negotiation.cc:407] Connection header received
0812 06:55:23.592635 (+ 483us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:55:23.592638 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:55:23.592697 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:55:23.592787 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:55:23.593524 (+ 737us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:23.594150 (+ 626us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:23.595347 (+ 1197us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:23.595584 (+ 237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:23.595705 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:55:23.598213 (+ 2508us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:55:23.598216 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:55:23.598218 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:55:23.598246 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:55:23.600037 (+ 1791us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:55:23.600680 (+ 643us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:55:23.600683 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:55:23.600685 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:55:23.600772 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:55:23.601127 (+ 355us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:55:23.601131 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:55:23.601132 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:55:23.601396 (+ 264us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:55:23.601501 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0812 06:55:23.601933 (+ 432us) server_negotiation.cc:299] Negotiation successful
0812 06:55:23.602074 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260812 06:55:23.602211 27423 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:23.591528 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.126:34933 (local address 127.0.0.1:44452)
0812 06:55:23.591996 (+ 468us) negotiation.cc:107] Waiting for socket to connect
0812 06:55:23.592050 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0812 06:55:23.592180 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:55:23.592880 (+ 700us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:55:23.592884 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:55:23.593248 (+ 364us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:55:23.593390 (+ 142us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:55:23.593396 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:55:23.594338 (+ 942us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:55:23.594342 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:55:23.595216 (+ 874us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:55:23.595226 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:55:23.595706 (+ 480us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:55:23.595710 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:55:23.595833 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:55:23.596416 (+ 583us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 06:55:23.596428 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:55:23.598044 (+ 1616us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 06:55:23.600184 (+ 2140us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:55:23.600191 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:55:23.600197 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:55:23.600527 (+ 330us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:55:23.600875 (+ 348us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:55:23.600878 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:55:23.600880 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:55:23.600975 (+ 95us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:55:23.601541 (+ 566us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 06:55:23.601548 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 06:55:23.601685 (+ 137us) client_negotiation.cc:769] Sending connection context
0812 06:55:23.601841 (+ 156us) client_negotiation.cc:240] Negotiation successful
0812 06:55:23.602027 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":302,"thread_start_us":204,"threads_started":1}
I20260812 06:55:23.603941 27419 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.19.52.125" port: 32823 } has no permanent_uuid. Determining permanent_uuid...
Aug 12 06:55:23 dist-test-slave-65mx krb5kdc[26753](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517723, etypes {rep=17 tkt=17 ses=17}, oryx/127.19.52.125@KRBTEST.COM for oryx/127.19.52.125@KRBTEST.COM
I20260812 06:55:23.613327 27422 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:23.604811 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55212 (local address 127.19.52.125:32823)
0812 06:55:23.604939 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0812 06:55:23.604943 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 06:55:23.604953 (+ 10us) server_negotiation.cc:407] Connection header received
0812 06:55:23.604978 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:55:23.604979 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:55:23.605022 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:55:23.605115 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:55:23.605529 (+ 414us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:23.606400 (+ 871us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:23.607215 (+ 815us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:23.607440 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:23.607552 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:55:23.609932 (+ 2380us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:55:23.609941 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:55:23.609943 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:55:23.609964 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:55:23.611544 (+ 1580us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:55:23.612005 (+ 461us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:55:23.612013 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:55:23.612014 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:55:23.612092 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:55:23.612376 (+ 284us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:55:23.612379 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:55:23.612381 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:55:23.612656 (+ 275us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:55:23.612776 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0812 06:55:23.613029 (+ 253us) server_negotiation.cc:299] Negotiation successful
0812 06:55:23.613192 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":62,"server-negotiator.queue_time_us":52}
I20260812 06:55:23.613327 27423 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:23.604526 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.125:32823 (local address 127.0.0.1:55212)
0812 06:55:23.604709 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0812 06:55:23.604724 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 06:55:23.604816 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:55:23.605127 (+ 311us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:55:23.605129 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:55:23.605331 (+ 202us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:55:23.605418 (+ 87us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:55:23.605423 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:55:23.606549 (+ 1126us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:55:23.606552 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:55:23.607091 (+ 539us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:55:23.607099 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:55:23.607562 (+ 463us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:55:23.607565 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:55:23.607651 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:55:23.608219 (+ 568us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 06:55:23.608247 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:55:23.609774 (+ 1527us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 06:55:23.611677 (+ 1903us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:55:23.611682 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:55:23.611684 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:55:23.611890 (+ 206us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:55:23.612185 (+ 295us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:55:23.612189 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:55:23.612191 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:55:23.612277 (+ 86us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:55:23.612764 (+ 487us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 06:55:23.612767 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 06:55:23.612862 (+ 95us) client_negotiation.cc:769] Sending connection context
0812 06:55:23.612974 (+ 112us) client_negotiation.cc:240] Negotiation successful
0812 06:55:23.613135 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":104}
I20260812 06:55:23.617014 27419 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2b5a510eb7be4cfab2f7914ebabaa929: Bootstrap starting.
I20260812 06:55:23.619488 27419 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2b5a510eb7be4cfab2f7914ebabaa929: Neither blocks nor log segments found. Creating new log.
I20260812 06:55:23.620352 27419 log.cc:824] T 00000000000000000000000000000000 P 2b5a510eb7be4cfab2f7914ebabaa929: Log is configured to *not* fsync() on all Append() calls
I20260812 06:55:23.622705 27419 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2b5a510eb7be4cfab2f7914ebabaa929: No bootstrap required, opened a new log
I20260812 06:55:23.626056 27419 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2b5a510eb7be4cfab2f7914ebabaa929 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "53e6ebc3b8ee4672983fb82b50148e26" member_type: VOTER last_known_addr { host: "127.19.52.126" port: 34933 } } peers { permanent_uuid: "2b5a510eb7be4cfab2f7914ebabaa929" member_type: VOTER last_known_addr { host: "127.19.52.125" port: 32823 } }
I20260812 06:55:23.626313 27419 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2b5a510eb7be4cfab2f7914ebabaa929 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 06:55:23.626350 27419 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2b5a510eb7be4cfab2f7914ebabaa929 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2b5a510eb7be4cfab2f7914ebabaa929, State: Initialized, Role: FOLLOWER
I20260812 06:55:23.626806 27419 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2b5a510eb7be4cfab2f7914ebabaa929 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "53e6ebc3b8ee4672983fb82b50148e26" member_type: VOTER last_known_addr { host: "127.19.52.126" port: 34933 } } peers { permanent_uuid: "2b5a510eb7be4cfab2f7914ebabaa929" member_type: VOTER last_known_addr { host: "127.19.52.125" port: 32823 } }
I20260812 06:55:23.627316 27425 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2b5a510eb7be4cfab2f7914ebabaa929 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "53e6ebc3b8ee4672983fb82b50148e26" member_type: VOTER last_known_addr { host: "127.19.52.126" port: 34933 } } peers { permanent_uuid: "2b5a510eb7be4cfab2f7914ebabaa929" member_type: VOTER last_known_addr { host: "127.19.52.125" port: 32823 } } }
I20260812 06:55:23.627486 27425 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2b5a510eb7be4cfab2f7914ebabaa929 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 06:55:23.628181 27419 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2b5a510eb7be4cfab2f7914ebabaa929 [sys.catalog]: configured and running, proceeding with master startup.
W20260812 06:55:23.634375 27437 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 2b5a510eb7be4cfab2f7914ebabaa929: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 06:55:23.634545 27437 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 06:55:23.637079 27340 master_runner.cc:428] Detected that this master 2b5a510eb7be4cfab2f7914ebabaa929 is joining an existing cluster
I20260812 06:55:23.637152 27340 master_runner.cc:432] Initiating AddMaster RPC to add 127.19.52.125:32823
I20260812 06:55:23.649438 26848 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:23.639776 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44458 (local address 127.19.52.126:34933)
0812 06:55:23.639906 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0812 06:55:23.639914 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 06:55:23.640491 (+ 577us) server_negotiation.cc:407] Connection header received
0812 06:55:23.640656 (+ 165us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:55:23.640659 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:55:23.640721 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:55:23.640816 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:55:23.641504 (+ 688us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:23.642324 (+ 820us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:23.643558 (+ 1234us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:23.643841 (+ 283us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:23.643952 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:55:23.645612 (+ 1660us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:55:23.645616 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:55:23.645619 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:55:23.645650 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:55:23.647482 (+ 1832us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:55:23.648014 (+ 532us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:55:23.648017 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:55:23.648019 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:55:23.648106 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:55:23.648399 (+ 293us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:55:23.648404 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:55:23.648406 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:55:23.648709 (+ 303us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:55:23.648813 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0812 06:55:23.649129 (+ 316us) server_negotiation.cc:299] Negotiation successful
0812 06:55:23.649275 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260812 06:55:23.649443 27444 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:23.639790 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.126:34933 (local address 127.0.0.1:44458)
0812 06:55:23.640353 (+ 563us) negotiation.cc:107] Waiting for socket to connect
0812 06:55:23.640373 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0812 06:55:23.640516 (+ 143us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:55:23.640837 (+ 321us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:55:23.640840 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:55:23.641151 (+ 311us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:55:23.641338 (+ 187us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:55:23.641346 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:55:23.642453 (+ 1107us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:55:23.642456 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:55:23.643419 (+ 963us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:55:23.643427 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:55:23.644103 (+ 676us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:55:23.644110 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:55:23.644243 (+ 133us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:55:23.644855 (+ 612us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 06:55:23.644875 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:55:23.645439 (+ 564us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 06:55:23.647612 (+ 2173us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:55:23.647618 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:55:23.647621 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:55:23.647891 (+ 270us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:55:23.648205 (+ 314us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:55:23.648208 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:55:23.648210 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:55:23.648304 (+ 94us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:55:23.648814 (+ 510us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 06:55:23.648817 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 06:55:23.648919 (+ 102us) client_negotiation.cc:769] Sending connection context
0812 06:55:23.649066 (+ 147us) client_negotiation.cc:240] Negotiation successful
0812 06:55:23.649223 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":454,"thread_start_us":188,"threads_started":1}
I20260812 06:55:23.649482 27422 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:23.639693 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55218 (local address 127.19.52.125:32823)
0812 06:55:23.639843 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0812 06:55:23.639848 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 06:55:23.640631 (+ 783us) server_negotiation.cc:407] Connection header received
0812 06:55:23.640789 (+ 158us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:55:23.640796 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:55:23.640850 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:55:23.640930 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:55:23.641561 (+ 631us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:23.642107 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:23.643400 (+ 1293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:23.643758 (+ 358us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:23.643876 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:55:23.645988 (+ 2112us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:55:23.645991 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:55:23.645994 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:55:23.646018 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:55:23.647808 (+ 1790us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:55:23.648252 (+ 444us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:55:23.648255 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:55:23.648257 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:55:23.648326 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:55:23.648636 (+ 310us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:55:23.648640 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:55:23.648642 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:55:23.648861 (+ 219us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:55:23.648990 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0812 06:55:23.649238 (+ 248us) server_negotiation.cc:299] Negotiation successful
0812 06:55:23.649342 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260812 06:55:23.649936 27443 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:23.639916 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.125:32823 (local address 127.0.0.1:55218)
0812 06:55:23.640526 (+ 610us) negotiation.cc:107] Waiting for socket to connect
0812 06:55:23.640564 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0812 06:55:23.640689 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:55:23.641060 (+ 371us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:55:23.641063 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:55:23.641296 (+ 233us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:55:23.641430 (+ 134us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:55:23.641454 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:55:23.642237 (+ 783us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:55:23.642240 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:55:23.643257 (+ 1017us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:55:23.643269 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:55:23.643916 (+ 647us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:55:23.643922 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:55:23.644016 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:55:23.644712 (+ 696us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 06:55:23.644724 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:55:23.645439 (+ 715us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 06:55:23.647921 (+ 2482us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:55:23.647924 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:55:23.647926 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:55:23.648157 (+ 231us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:55:23.648417 (+ 260us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:55:23.648420 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:55:23.648422 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:55:23.648514 (+ 92us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:55:23.648984 (+ 470us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 06:55:23.648987 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 06:55:23.649076 (+ 89us) client_negotiation.cc:769] Sending connection context
0812 06:55:23.649622 (+ 546us) client_negotiation.cc:240] Negotiation successful
0812 06:55:23.649790 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":539,"mutex_wait_us":36,"spinlock_wait_cycles":21376,"thread_start_us":188,"threads_started":1}
Aug 12 06:55:23 dist-test-slave-65mx krb5kdc[26753](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517722, etypes {rep=17 tkt=17 ses=17}, oryx/127.19.52.126@KRBTEST.COM for oryx/127.19.52.125@KRBTEST.COM
WARNING: no policy specified for oryx/127.19.52.124@KRBTEST.COM; defaulting to no policy
I20260812 06:55:23.668977 27445 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:23.655590 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.125:32823 (local address 127.0.0.1:55234)
0812 06:55:23.656066 (+ 476us) negotiation.cc:107] Waiting for socket to connect
0812 06:55:23.656134 (+ 68us) client_negotiation.cc:174] Beginning negotiation
0812 06:55:23.656307 (+ 173us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:55:23.659068 (+ 2761us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:55:23.659078 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:55:23.659741 (+ 663us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:55:23.659896 (+ 155us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:55:23.659904 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:55:23.661077 (+ 1173us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:55:23.661081 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:55:23.661905 (+ 824us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:55:23.661918 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:55:23.662570 (+ 652us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:55:23.662573 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:55:23.662680 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:55:23.663436 (+ 756us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 06:55:23.663449 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:55:23.665017 (+ 1568us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 06:55:23.666914 (+ 1897us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:55:23.666925 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:55:23.666933 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:55:23.667246 (+ 313us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:55:23.667551 (+ 305us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:55:23.667554 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:55:23.667557 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:55:23.667661 (+ 104us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:55:23.668153 (+ 492us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 06:55:23.668163 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 06:55:23.668352 (+ 189us) client_negotiation.cc:769] Sending connection context
0812 06:55:23.668551 (+ 199us) client_negotiation.cc:240] Negotiation successful
0812 06:55:23.668813 (+ 262us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":247,"thread_start_us":115,"threads_started":1}
I20260812 06:55:23.668972 27422 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:23.658252 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55234 (local address 127.19.52.125:32823)
0812 06:55:23.658413 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0812 06:55:23.658418 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 06:55:23.658557 (+ 139us) server_negotiation.cc:407] Connection header received
0812 06:55:23.658664 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:55:23.658669 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:55:23.658750 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:55:23.658855 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:55:23.660049 (+ 1194us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:23.660942 (+ 893us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:23.662043 (+ 1101us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:23.662312 (+ 269us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:23.662429 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:55:23.665199 (+ 2770us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:55:23.665203 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:55:23.665205 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:55:23.665231 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:55:23.666766 (+ 1535us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:55:23.667347 (+ 581us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:55:23.667351 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:55:23.667353 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:55:23.667445 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:55:23.667762 (+ 317us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:55:23.667767 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:55:23.667769 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:55:23.668016 (+ 247us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:55:23.668139 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0812 06:55:23.668640 (+ 501us) server_negotiation.cc:299] Negotiation successful
0812 06:55:23.668795 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.19.52.124@KRBTEST.COM" created.
I20260812 06:55:23.670810 26802 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.19.52.125:32823: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "2b5a510eb7be4cfab2f7914ebabaa929"
member_type: NON_VOTER
last_known_addr {
host: "127.19.52.125"
port: 32823
}
attrs {
promote: true
}
}
dest_uuid: "53e6ebc3b8ee4672983fb82b50148e26"
cas_config_opid_index: -1
I20260812 06:55:23.671720 26802 consensus_queue.cc:237] T 00000000000000000000000000000000 P 53e6ebc3b8ee4672983fb82b50148e26 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "53e6ebc3b8ee4672983fb82b50148e26" member_type: VOTER last_known_addr { host: "127.19.52.126" port: 34933 } } peers { permanent_uuid: "2b5a510eb7be4cfab2f7914ebabaa929" member_type: NON_VOTER last_known_addr { host: "127.19.52.125" port: 32823 } attrs { promote: true } }
I20260812 06:55:23.674027 27452 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 53e6ebc3b8ee4672983fb82b50148e26 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER 2b5a510eb7be4cfab2f7914ebabaa929 (127.19.52.125) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "53e6ebc3b8ee4672983fb82b50148e26" member_type: VOTER last_known_addr { host: "127.19.52.126" port: 34933 } } peers { permanent_uuid: "2b5a510eb7be4cfab2f7914ebabaa929" member_type: NON_VOTER last_known_addr { host: "127.19.52.125" port: 32823 } attrs { promote: true } } }
I20260812 06:55:23.674901 27388 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2b5a510eb7be4cfab2f7914ebabaa929 [term 0 FOLLOWER]: Advancing to term 1
I20260812 06:55:23.676064 27452 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.19.52.125:32823
I20260812 06:55:23.676487 27453 sys_catalog.cc:455] T 00000000000000000000000000000000 P 53e6ebc3b8ee4672983fb82b50148e26 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "53e6ebc3b8ee4672983fb82b50148e26" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "53e6ebc3b8ee4672983fb82b50148e26" member_type: VOTER last_known_addr { host: "127.19.52.126" port: 34933 } } peers { permanent_uuid: "2b5a510eb7be4cfab2f7914ebabaa929" member_type: NON_VOTER last_known_addr { host: "127.19.52.125" port: 32823 } attrs { promote: true } } }
I20260812 06:55:23.676654 27388 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 2b5a510eb7be4cfab2f7914ebabaa929 [term 1 FOLLOWER]: Refusing update from remote peer 53e6ebc3b8ee4672983fb82b50148e26: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20260812 06:55:23.676733 27340 master.cc:537] Master@127.19.52.125:32823 shutting down...
I20260812 06:55:23.677615 27453 sys_catalog.cc:458] T 00000000000000000000000000000000 P 53e6ebc3b8ee4672983fb82b50148e26 [sys.catalog]: This master's current role is: LEADER
I20260812 06:55:23.677407 27452 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 53e6ebc3b8ee4672983fb82b50148e26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2b5a510eb7be4cfab2f7914ebabaa929" member_type: NON_VOTER last_known_addr { host: "127.19.52.125" port: 32823 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 0, Time since last communication: 0.000s
W20260812 06:55:23.680029 26790 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master
W20260812 06:55:23.681149 26790 consensus_peers.cc:487] T 00000000000000000000000000000000 P 53e6ebc3b8ee4672983fb82b50148e26 -> Peer 2b5a510eb7be4cfab2f7914ebabaa929 (127.19.52.125:32823): Couldn't send request to peer 2b5a510eb7be4cfab2f7914ebabaa929. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260812 06:55:23.682566 27340 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 2b5a510eb7be4cfab2f7914ebabaa929 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 06:55:23.682883 27340 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 2b5a510eb7be4cfab2f7914ebabaa929 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 06:55:23.683012 27340 tablet_replica.cc:330] T 00000000000000000000000000000000 P 2b5a510eb7be4cfab2f7914ebabaa929: stopping tablet replica
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.19.52.124 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517687384214-19665-0/krb5kdc/oryx_127.19.52.124.keytab.
Entry for principal oryx/127.19.52.124 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517687384214-19665-0/krb5kdc/oryx_127.19.52.124.keytab.
I20260812 06:55:23.696135 19665 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu
/tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517687384214-19665-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517687384214-19665-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517687384214-19665-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517687384214-19665-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.52.124:45647
--webserver_interface=127.19.52.124
--webserver_port=0
--keytab_file=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517687384214-19665-0/krb5kdc/oryx_127.19.52.124.keytab
--principal=oryx/127.19.52.124
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.52.84:43707
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.19.52.126:34933,127.19.52.125:32823,127.19.52.124:45647
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517687384214-19665-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517687384214-19665-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517687384214-19665-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 06:55:23.698954 27340 master.cc:552] Master@127.19.52.125:32823 shutdown complete.
I20260812 06:55:23.699051 27340 master_runner.cc:305] Clearing existing system tablet
I20260812 06:55:23.699311 27340 fs_manager.cc:359] Metadata directory not provided
I20260812 06:55:23.699390 27340 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:55:23.700687 27340 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260812 06:55:23.701658 27460 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 06:55:23.701923 27340 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517687384214-19665-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:55:23.702052 27340 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 06:55:23.702087 27340 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517687384214-19665-0/minicluster-data/master-1/data,/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517687384214-19665-0/minicluster-data/master-1/wal
uuid: "2b5a510eb7be4cfab2f7914ebabaa929"
format_stamp: "Formatted at 2026-08-12 06:55:23 on dist-test-slave-65mx"
server_key: "63146d8807a10f93ad7c727bafd0367c"
server_key_iv: "27b0bd8d4674412ac6353b77d07bb687"
server_key_version: "encryptionkey@0"
I20260812 06:55:23.704339 27340 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P 2b5a510eb7be4cfab2f7914ebabaa929: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 06:55:23.706807 27340 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P 2b5a510eb7be4cfab2f7914ebabaa929: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260812 06:55:23.706907 27340 log.cc:1196] T 00000000000000000000000000000000 P 2b5a510eb7be4cfab2f7914ebabaa929: Deleting WAL directory at /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517687384214-19665-0/minicluster-data/master-1/wal/wals/00000000000000000000000000000000
I20260812 06:55:23.707201 27340 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P 2b5a510eb7be4cfab2f7914ebabaa929: Deleting consensus metadata
I20260812 06:55:23.707402 27340 master_runner.cc:315] Copying system tablet from 127.19.52.126:34933
I20260812 06:55:23.708841 27340 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P 2b5a510eb7be4cfab2f7914ebabaa929: tablet copy: Beginning tablet copy session from remote peer at address 127.19.52.126:34933
I20260812 06:55:23.718652 26848 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:23.709636 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44470 (local address 127.19.52.126:34933)
0812 06:55:23.709789 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0812 06:55:23.709794 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 06:55:23.710048 (+ 254us) server_negotiation.cc:407] Connection header received
0812 06:55:23.710190 (+ 142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:55:23.710193 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:55:23.710249 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:55:23.710330 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:55:23.710859 (+ 529us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:23.711713 (+ 854us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:23.712925 (+ 1212us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:23.713287 (+ 362us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:23.713394 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:55:23.714810 (+ 1416us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:55:23.714814 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:55:23.714817 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:55:23.714846 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:55:23.716667 (+ 1821us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:55:23.717196 (+ 529us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:55:23.717199 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:55:23.717201 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:55:23.717291 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:55:23.717659 (+ 368us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:55:23.717663 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:55:23.717665 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:55:23.717928 (+ 263us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:55:23.718045 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0812 06:55:23.718349 (+ 304us) server_negotiation.cc:299] Negotiation successful
0812 06:55:23.718486 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260812 06:55:23.718669 27466 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:23.709543 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.126:34933 (local address 127.0.0.1:44470)
0812 06:55:23.709930 (+ 387us) negotiation.cc:107] Waiting for socket to connect
0812 06:55:23.709945 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 06:55:23.710086 (+ 141us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:55:23.710368 (+ 282us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:55:23.710371 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:55:23.710601 (+ 230us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:55:23.710734 (+ 133us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:55:23.710741 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:55:23.711840 (+ 1099us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:55:23.711843 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:55:23.712790 (+ 947us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:55:23.712801 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:55:23.713421 (+ 620us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:55:23.713425 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:55:23.713536 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:55:23.714140 (+ 604us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 06:55:23.714156 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:55:23.714646 (+ 490us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 06:55:23.716791 (+ 2145us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:55:23.716795 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:55:23.716797 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:55:23.717083 (+ 286us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:55:23.717397 (+ 314us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:55:23.717402 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:55:23.717404 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:55:23.717504 (+ 100us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:55:23.718047 (+ 543us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 06:55:23.718050 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 06:55:23.718159 (+ 109us) client_negotiation.cc:769] Sending connection context
0812 06:55:23.718272 (+ 113us) client_negotiation.cc:240] Negotiation successful
0812 06:55:23.718429 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":303,"thread_start_us":89,"threads_started":1}
I20260812 06:55:23.719842 26825 tablet_copy_service.cc:140] P 53e6ebc3b8ee4672983fb82b50148e26: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer 2b5a510eb7be4cfab2f7914ebabaa929 ({username='oryx', principal='oryx/127.19.52.125@KRBTEST.COM'} at 127.0.0.1:44470)
I20260812 06:55:23.720043 26825 tablet_copy_service.cc:161] P 53e6ebc3b8ee4672983fb82b50148e26: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer 2b5a510eb7be4cfab2f7914ebabaa929 at {username='oryx', principal='oryx/127.19.52.125@KRBTEST.COM'} at 127.0.0.1:44470: session id = 2b5a510eb7be4cfab2f7914ebabaa929-00000000000000000000000000000000
I20260812 06:55:23.722293 26825 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P 53e6ebc3b8ee4672983fb82b50148e26: Tablet Copy: opened 0 blocks and 1 log segments
I20260812 06:55:23.725194 27340 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260812 06:55:23.727677 27340 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P 2b5a510eb7be4cfab2f7914ebabaa929: tablet copy: Starting download of 0 data blocks...
I20260812 06:55:23.727878 27340 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P 2b5a510eb7be4cfab2f7914ebabaa929: tablet copy: Starting download of 1 WAL segments...
I20260812 06:55:23.732070 27340 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P 2b5a510eb7be4cfab2f7914ebabaa929: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260812 06:55:23.734148 26825 tablet_copy_service.cc:342] P 53e6ebc3b8ee4672983fb82b50148e26: Request end of tablet copy session 2b5a510eb7be4cfab2f7914ebabaa929-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.19.52.125@KRBTEST.COM'} at 127.0.0.1:44470
I20260812 06:55:23.734295 26825 tablet_copy_service.cc:433] P 53e6ebc3b8ee4672983fb82b50148e26: ending tablet copy session 2b5a510eb7be4cfab2f7914ebabaa929-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer 2b5a510eb7be4cfab2f7914ebabaa929
I20260812 06:55:23.743731 27340 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 06:55:23.745834 27467 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:55:23.750021 27340 server_base.cc:1034] running on GCE node
W20260812 06:55:23.750164 27470 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:55:23.753317 27468 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:55:23.753641 27340 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:55:23.754014 27340 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:55:23.755245 27340 hybrid_clock.cc:648] HybridClock initialized: now 1786517723755212 us; error 55 us; skew 500 ppm
Aug 12 06:55:23 dist-test-slave-65mx krb5kdc[26753](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517723, etypes {rep=17 tkt=17 ses=17}, oryx/127.19.52.125@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 06:55:23.757412 27340 init.cc:377] Logged in from keytab as oryx/127.19.52.125@KRBTEST.COM (short username oryx)
I20260812 06:55:23.758010 27340 webserver.cc:460] Webserver started at http://127.19.52.125:44059/ using document root <none> and password file <none>
I20260812 06:55:23.758229 27340 fs_manager.cc:359] Metadata directory not provided
I20260812 06:55:23.758276 27340 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:55:23.760010 27340 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.002s
I20260812 06:55:23.761204 27481 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 06:55:23.761754 27340 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260812 06:55:23.761849 27340 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517687384214-19665-0/minicluster-data/master-1/data,/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517687384214-19665-0/minicluster-data/master-1/wal
uuid: "2b5a510eb7be4cfab2f7914ebabaa929"
format_stamp: "Formatted at 2026-08-12 06:55:23 on dist-test-slave-65mx"
server_key: "63146d8807a10f93ad7c727bafd0367c"
server_key_iv: "27b0bd8d4674412ac6353b77d07bb687"
server_key_version: "encryptionkey@0"
I20260812 06:55:23.761945 27340 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517687384214-19665-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517687384214-19665-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517687384214-19665-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:55:23.775882 27340 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 06:55:23.778257 27340 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:55:23.784123 27340 rpc_server.cc:307] RPC server started. Bound to: 127.19.52.125:32823
I20260812 06:55:23.784140 27544 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.52.125:32823 every 8 connection(s)
I20260812 06:55:23.784992 27340 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517687384214-19665-0/minicluster-data/master-1/data/info.pb
I20260812 06:55:23.787036 27545 sys_catalog.cc:263] Verifying existing consensus state
I20260812 06:55:23.788169 27545 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2b5a510eb7be4cfab2f7914ebabaa929: Bootstrap starting.
I20260812 06:55:23.801043 27545 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2b5a510eb7be4cfab2f7914ebabaa929: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260812 06:55:23.801630 27545 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2b5a510eb7be4cfab2f7914ebabaa929: Bootstrap complete.
I20260812 06:55:23.802407 27545 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2b5a510eb7be4cfab2f7914ebabaa929 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "53e6ebc3b8ee4672983fb82b50148e26" member_type: VOTER last_known_addr { host: "127.19.52.126" port: 34933 } } peers { permanent_uuid: "2b5a510eb7be4cfab2f7914ebabaa929" member_type: NON_VOTER last_known_addr { host: "127.19.52.125" port: 32823 } attrs { promote: true } }
I20260812 06:55:23.802580 27545 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2b5a510eb7be4cfab2f7914ebabaa929 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 2b5a510eb7be4cfab2f7914ebabaa929, State: Initialized, Role: LEARNER
I20260812 06:55:23.802743 27545 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2b5a510eb7be4cfab2f7914ebabaa929 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "53e6ebc3b8ee4672983fb82b50148e26" member_type: VOTER last_known_addr { host: "127.19.52.126" port: 34933 } } peers { permanent_uuid: "2b5a510eb7be4cfab2f7914ebabaa929" member_type: NON_VOTER last_known_addr { host: "127.19.52.125" port: 32823 } attrs { promote: true } }
I20260812 06:55:23.803176 27548 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2b5a510eb7be4cfab2f7914ebabaa929 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "53e6ebc3b8ee4672983fb82b50148e26" member_type: VOTER last_known_addr { host: "127.19.52.126" port: 34933 } } peers { permanent_uuid: "2b5a510eb7be4cfab2f7914ebabaa929" member_type: NON_VOTER last_known_addr { host: "127.19.52.125" port: 32823 } attrs { promote: true } } }
I20260812 06:55:23.803290 27545 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2b5a510eb7be4cfab2f7914ebabaa929 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 06:55:23.803411 27548 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2b5a510eb7be4cfab2f7914ebabaa929 [sys.catalog]: This master's current role is: LEARNER
I20260812 06:55:23.806907 27561 catalog_manager.cc:1259] Loaded cluster ID: 020dffafdb2c4d38b19668de7f5df860
I20260812 06:55:23.807119 27561 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 2b5a510eb7be4cfab2f7914ebabaa929: loading cluster ID for follower catalog manager: success
I20260812 06:55:23.809671 27561 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 2b5a510eb7be4cfab2f7914ebabaa929: acquiring CA information for follower catalog manager: success
I20260812 06:55:23.811314 27561 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 2b5a510eb7be4cfab2f7914ebabaa929: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260812 06:55:23.843436 27020 heartbeater.cc:503] Master 127.19.52.126:34933 was elected leader, sending a full tablet report...
W20260812 06:55:23.843137 27456 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:55:23.843559 27456 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:55:23.843619 27456 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 06:55:23.848166 27456 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 06:55:23.848245 27456 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 06:55:23.848266 27456 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 06:55:23.848284 27456 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 06:55:23.848299 27456 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 06:55:23.854032 27456 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.52.84:43707
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517687384214-19665-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517687384214-19665-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.19.52.126:34933,127.19.52.125:32823,127.19.52.124:45647
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517687384214-19665-0/krb5kdc/oryx_127.19.52.124.keytab
--principal=oryx/127.19.52.124
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.52.124:45647
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517687384214-19665-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.19.52.124
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27456
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517687384214-19665-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 06:15:54 UTC on ccycloud.quasar-rnfzce.root.comops.site
build id 4581989
I20260812 06:55:23.855202 27456 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:55:23.856271 27456 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 06:55:23.867276 27567 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:55:23.867264 27566 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:55:23.867271 27569 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:55:23.867594 27456 server_base.cc:1034] running on GCE node
I20260812 06:55:23.868387 27456 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:55:23.869854 27456 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:55:23.871070 27456 hybrid_clock.cc:648] HybridClock initialized: now 1786517723871050 us; error 40 us; skew 500 ppm
Aug 12 06:55:23 dist-test-slave-65mx krb5kdc[26753](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517723, etypes {rep=17 tkt=17 ses=17}, oryx/127.19.52.124@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 06:55:23.874590 27456 init.cc:377] Logged in from keytab as oryx/127.19.52.124@KRBTEST.COM (short username oryx)
I20260812 06:55:23.875666 27456 webserver.cc:460] Webserver started at http://127.19.52.124:35789/ using document root <none> and password file <none>
I20260812 06:55:23.876263 27456 fs_manager.cc:359] Metadata directory not provided
I20260812 06:55:23.876364 27456 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:55:23.876679 27456 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 06:55:23.878870 27456 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517687384214-19665-0/minicluster-data/master-2/data/instance:
uuid: "950b5d35439b4e23bd6d86c6664ddc43"
format_stamp: "Formatted at 2026-08-12 06:55:23 on dist-test-slave-65mx"
server_key: "f953733894525dc31a97d6d4b4758319"
server_key_iv: "25088ae180f6f66f48b8976c89c7ced9"
server_key_version: "encryptionkey@0"
I20260812 06:55:23.879455 27456 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517687384214-19665-0/minicluster-data/master-2/wal/instance:
uuid: "950b5d35439b4e23bd6d86c6664ddc43"
format_stamp: "Formatted at 2026-08-12 06:55:23 on dist-test-slave-65mx"
server_key: "f953733894525dc31a97d6d4b4758319"
server_key_iv: "25088ae180f6f66f48b8976c89c7ced9"
server_key_version: "encryptionkey@0"
I20260812 06:55:23.883606 27456 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.004s
I20260812 06:55:23.886433 27576 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 06:55:23.887888 27456 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260812 06:55:23.888089 27456 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517687384214-19665-0/minicluster-data/master-2/data,/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517687384214-19665-0/minicluster-data/master-2/wal
uuid: "950b5d35439b4e23bd6d86c6664ddc43"
format_stamp: "Formatted at 2026-08-12 06:55:23 on dist-test-slave-65mx"
server_key: "f953733894525dc31a97d6d4b4758319"
server_key_iv: "25088ae180f6f66f48b8976c89c7ced9"
server_key_version: "encryptionkey@0"
I20260812 06:55:23.888221 27456 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517687384214-19665-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517687384214-19665-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517687384214-19665-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:55:23.910076 27456 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 06:55:23.913622 27456 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 06:55:23.913853 27456 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:55:23.922188 27456 rpc_server.cc:307] RPC server started. Bound to: 127.19.52.124:45647
I20260812 06:55:23.922494 27639 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.52.124:45647 every 8 connection(s)
I20260812 06:55:23.923401 27456 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517687384214-19665-0/minicluster-data/master-2/data/info.pb
I20260812 06:55:23.926448 27640 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 06:55:23.927943 19665 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu as pid 27456
I20260812 06:55:23.930689 27640 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.19.52.126" port: 34933 } has no permanent_uuid. Determining permanent_uuid...
Aug 12 06:55:23 dist-test-slave-65mx krb5kdc[26753](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517722, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.19.52.124@KRBTEST.COM
Aug 12 06:55:23 dist-test-slave-65mx krb5kdc[26753](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517723, etypes {rep=17 tkt=17 ses=17}, oryx/127.19.52.124@KRBTEST.COM for oryx/127.19.52.126@KRBTEST.COM
I20260812 06:55:23.943634 26848 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:23.933583 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44472 (local address 127.19.52.126:34933)
0812 06:55:23.933737 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0812 06:55:23.933742 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 06:55:23.933962 (+ 220us) server_negotiation.cc:407] Connection header received
0812 06:55:23.934139 (+ 177us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:55:23.934143 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:55:23.934198 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:55:23.934280 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:55:23.935115 (+ 835us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:23.935763 (+ 648us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:23.936970 (+ 1207us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:23.937353 (+ 383us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:23.937499 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:55:23.939977 (+ 2478us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:55:23.939982 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:55:23.939989 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:55:23.940014 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:55:23.941607 (+ 1593us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:55:23.942115 (+ 508us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:55:23.942119 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:55:23.942120 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:55:23.942206 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:55:23.942491 (+ 285us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:55:23.942496 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:55:23.942498 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:55:23.942687 (+ 189us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:55:23.942797 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0812 06:55:23.943160 (+ 363us) server_negotiation.cc:299] Negotiation successful
0812 06:55:23.943461 (+ 301us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260812 06:55:23.944335 27643 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:23.933429 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.126:34933 (local address 127.0.0.1:44472)
0812 06:55:23.933842 (+ 413us) negotiation.cc:107] Waiting for socket to connect
0812 06:55:23.933883 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0812 06:55:23.934017 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:55:23.934332 (+ 315us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:55:23.934337 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:55:23.934796 (+ 459us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:55:23.934953 (+ 157us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:55:23.934962 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:55:23.935902 (+ 940us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:55:23.935905 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:55:23.936848 (+ 943us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:55:23.936857 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:55:23.937552 (+ 695us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:55:23.937556 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:55:23.937642 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:55:23.938206 (+ 564us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 06:55:23.938223 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:55:23.939825 (+ 1602us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 06:55:23.941748 (+ 1923us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:55:23.941756 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:55:23.941767 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:55:23.942006 (+ 239us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:55:23.942305 (+ 299us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:55:23.942308 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:55:23.942310 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:55:23.942397 (+ 87us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:55:23.942797 (+ 400us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 06:55:23.942805 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 06:55:23.942945 (+ 140us) client_negotiation.cc:769] Sending connection context
0812 06:55:23.943125 (+ 180us) client_negotiation.cc:240] Negotiation successful
0812 06:55:23.943517 (+ 392us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":247,"mutex_wait_us":135,"thread_start_us":138,"threads_started":1}
I20260812 06:55:23.944340 27642 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:23.929619 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36270 (local address 127.19.52.124:45647)
0812 06:55:23.930215 (+ 596us) server_negotiation.cc:206] Beginning negotiation
0812 06:55:23.930240 (+ 25us) server_negotiation.cc:399] Waiting for connection header
0812 06:55:23.930321 (+ 81us) server_negotiation.cc:407] Connection header received
0812 06:55:23.931183 (+ 862us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:55:23.931203 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:55:23.931585 (+ 382us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:55:23.931915 (+ 330us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:55:23.933095 (+ 1180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:23.934459 (+ 1364us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:23.935513 (+ 1054us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:23.936160 (+ 647us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:23.936313 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:55:23.939303 (+ 2990us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:55:23.939309 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:55:23.939320 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:55:23.939343 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:55:23.941382 (+ 2039us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:55:23.941860 (+ 478us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:55:23.941866 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:55:23.941867 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:55:23.941980 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:55:23.942277 (+ 297us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:55:23.942281 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:55:23.942283 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:55:23.942698 (+ 415us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:55:23.942929 (+ 231us) server_negotiation.cc:1035] Waiting for connection context
0812 06:55:23.943118 (+ 189us) server_negotiation.cc:299] Negotiation successful
0812 06:55:23.943434 (+ 316us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":5,"server-negotiator.queue_time_us":395,"thread_start_us":188,"threads_started":1}
I20260812 06:55:23.945746 27640 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.19.52.125" port: 32823 } has no permanent_uuid. Determining permanent_uuid...
Aug 12 06:55:23 dist-test-slave-65mx krb5kdc[26753](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517723, etypes {rep=17 tkt=17 ses=17}, oryx/127.19.52.124@KRBTEST.COM for oryx/127.19.52.125@KRBTEST.COM
I20260812 06:55:23.955809 27643 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:23.946380 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.125:32823 (local address 127.0.0.1:55250)
0812 06:55:23.946568 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0812 06:55:23.946587 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0812 06:55:23.946667 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:55:23.947010 (+ 343us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:55:23.947012 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:55:23.947185 (+ 173us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:55:23.947278 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:55:23.947283 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:55:23.948775 (+ 1492us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:55:23.948780 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:55:23.949423 (+ 643us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:55:23.949432 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:55:23.949889 (+ 457us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:55:23.949891 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:55:23.949966 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:55:23.950461 (+ 495us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 06:55:23.950473 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:55:23.951918 (+ 1445us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 06:55:23.954035 (+ 2117us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:55:23.954038 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:55:23.954040 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:55:23.954263 (+ 223us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:55:23.954685 (+ 422us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:55:23.954692 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:55:23.954694 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:55:23.954776 (+ 82us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:55:23.955225 (+ 449us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 06:55:23.955228 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 06:55:23.955341 (+ 113us) client_negotiation.cc:769] Sending connection context
0812 06:55:23.955456 (+ 115us) client_negotiation.cc:240] Negotiation successful
0812 06:55:23.955623 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":112}
I20260812 06:55:23.955834 27645 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:23.946503 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55250 (local address 127.19.52.125:32823)
0812 06:55:23.946800 (+ 297us) server_negotiation.cc:206] Beginning negotiation
0812 06:55:23.946804 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 06:55:23.946819 (+ 15us) server_negotiation.cc:407] Connection header received
0812 06:55:23.946859 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:55:23.946862 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:55:23.946904 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:55:23.946992 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:55:23.947383 (+ 391us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:23.948639 (+ 1256us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:23.949538 (+ 899us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:23.949785 (+ 247us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:23.949876 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:55:23.952110 (+ 2234us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:55:23.952114 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:55:23.952116 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:55:23.952143 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:55:23.953857 (+ 1714us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:55:23.954437 (+ 580us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:55:23.954440 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:55:23.954442 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:55:23.954544 (+ 102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:55:23.954881 (+ 337us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:55:23.954885 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:55:23.954887 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:55:23.955116 (+ 229us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:55:23.955235 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0812 06:55:23.955544 (+ 309us) server_negotiation.cc:299] Negotiation successful
0812 06:55:23.955682 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":222,"thread_start_us":110,"threads_started":1}
I20260812 06:55:23.957010 27640 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.19.52.124" port: 45647 } has no permanent_uuid. Determining permanent_uuid...
Aug 12 06:55:23 dist-test-slave-65mx krb5kdc[26753](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517723, etypes {rep=17 tkt=17 ses=17}, oryx/127.19.52.124@KRBTEST.COM for oryx/127.19.52.124@KRBTEST.COM
I20260812 06:55:23.967551 27643 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:23.957651 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.124:45647 (local address 127.0.0.1:36276)
0812 06:55:23.957819 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0812 06:55:23.957841 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0812 06:55:23.957912 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:55:23.958199 (+ 287us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:55:23.958201 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:55:23.958349 (+ 148us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:55:23.958449 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:55:23.958454 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:55:23.959798 (+ 1344us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:55:23.959801 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:55:23.960319 (+ 518us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:55:23.960328 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:55:23.961304 (+ 976us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:55:23.961308 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:55:23.961376 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:55:23.962103 (+ 727us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 06:55:23.962115 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:55:23.963894 (+ 1779us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 06:55:23.965822 (+ 1928us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:55:23.965825 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:55:23.965826 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:55:23.966104 (+ 278us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:55:23.966387 (+ 283us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:55:23.966389 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:55:23.966390 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:55:23.966500 (+ 110us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:55:23.966925 (+ 425us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 06:55:23.966928 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 06:55:23.967095 (+ 167us) client_negotiation.cc:769] Sending connection context
0812 06:55:23.967218 (+ 123us) client_negotiation.cc:240] Negotiation successful
0812 06:55:23.967362 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":76}
I20260812 06:55:23.967557 27642 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:23.957744 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36276 (local address 127.19.52.124:45647)
0812 06:55:23.957873 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0812 06:55:23.957885 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0812 06:55:23.957898 (+ 13us) server_negotiation.cc:407] Connection header received
0812 06:55:23.958018 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:55:23.958021 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:55:23.958065 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:55:23.958147 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:55:23.958603 (+ 456us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:23.959169 (+ 566us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:23.960646 (+ 1477us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:23.960929 (+ 283us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:23.961315 (+ 386us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:55:23.964132 (+ 2817us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:55:23.964135 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:55:23.964137 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:55:23.964161 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:55:23.965684 (+ 1523us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:55:23.966210 (+ 526us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:55:23.966212 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:55:23.966214 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:55:23.966293 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:55:23.966604 (+ 311us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:55:23.966607 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:55:23.966608 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:55:23.966804 (+ 196us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:55:23.966887 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0812 06:55:23.967279 (+ 392us) server_negotiation.cc:299] Negotiation successful
0812 06:55:23.967434 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":32,"server-negotiator.queue_time_us":33}
I20260812 06:55:23.970847 27640 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 950b5d35439b4e23bd6d86c6664ddc43: Bootstrap starting.
I20260812 06:55:23.973308 27640 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 950b5d35439b4e23bd6d86c6664ddc43: Neither blocks nor log segments found. Creating new log.
I20260812 06:55:23.974067 27640 log.cc:824] T 00000000000000000000000000000000 P 950b5d35439b4e23bd6d86c6664ddc43: Log is configured to *not* fsync() on all Append() calls
I20260812 06:55:23.976281 27640 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 950b5d35439b4e23bd6d86c6664ddc43: No bootstrap required, opened a new log
I20260812 06:55:23.979820 27640 raft_consensus.cc:348] T 00000000000000000000000000000000 P 950b5d35439b4e23bd6d86c6664ddc43 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "53e6ebc3b8ee4672983fb82b50148e26" member_type: VOTER last_known_addr { host: "127.19.52.126" port: 34933 } } peers { permanent_uuid: "2b5a510eb7be4cfab2f7914ebabaa929" member_type: VOTER last_known_addr { host: "127.19.52.125" port: 32823 } } peers { permanent_uuid: "950b5d35439b4e23bd6d86c6664ddc43" member_type: VOTER last_known_addr { host: "127.19.52.124" port: 45647 } }
I20260812 06:55:23.980068 27640 raft_consensus.cc:374] T 00000000000000000000000000000000 P 950b5d35439b4e23bd6d86c6664ddc43 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 06:55:23.980147 27640 raft_consensus.cc:729] T 00000000000000000000000000000000 P 950b5d35439b4e23bd6d86c6664ddc43 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 950b5d35439b4e23bd6d86c6664ddc43, State: Initialized, Role: FOLLOWER
I20260812 06:55:23.980893 27640 consensus_queue.cc:260] T 00000000000000000000000000000000 P 950b5d35439b4e23bd6d86c6664ddc43 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "53e6ebc3b8ee4672983fb82b50148e26" member_type: VOTER last_known_addr { host: "127.19.52.126" port: 34933 } } peers { permanent_uuid: "2b5a510eb7be4cfab2f7914ebabaa929" member_type: VOTER last_known_addr { host: "127.19.52.125" port: 32823 } } peers { permanent_uuid: "950b5d35439b4e23bd6d86c6664ddc43" member_type: VOTER last_known_addr { host: "127.19.52.124" port: 45647 } }
I20260812 06:55:23.981854 27647 sys_catalog.cc:455] T 00000000000000000000000000000000 P 950b5d35439b4e23bd6d86c6664ddc43 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "53e6ebc3b8ee4672983fb82b50148e26" member_type: VOTER last_known_addr { host: "127.19.52.126" port: 34933 } } peers { permanent_uuid: "2b5a510eb7be4cfab2f7914ebabaa929" member_type: VOTER last_known_addr { host: "127.19.52.125" port: 32823 } } peers { permanent_uuid: "950b5d35439b4e23bd6d86c6664ddc43" member_type: VOTER last_known_addr { host: "127.19.52.124" port: 45647 } } }
I20260812 06:55:23.982025 27647 sys_catalog.cc:458] T 00000000000000000000000000000000 P 950b5d35439b4e23bd6d86c6664ddc43 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 06:55:23.982448 27640 sys_catalog.cc:564] T 00000000000000000000000000000000 P 950b5d35439b4e23bd6d86c6664ddc43 [sys.catalog]: configured and running, proceeding with master startup.
W20260812 06:55:23.987811 27660 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 950b5d35439b4e23bd6d86c6664ddc43: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 06:55:23.987924 27660 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 06:55:23.991403 27456 master_runner.cc:428] Detected that this master 950b5d35439b4e23bd6d86c6664ddc43 is joining an existing cluster
I20260812 06:55:23.991487 27456 master_runner.cc:432] Initiating AddMaster RPC to add 127.19.52.124:45647
I20260812 06:55:23.997720 19665 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu with pid 26875
I20260812 06:55:24.013085 19665 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu
/tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517687384214-19665-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517687384214-19665-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517687384214-19665-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517687384214-19665-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.52.65:41417
--local_ip_for_outbound_sockets=127.19.52.65
--tserver_master_addrs=127.19.52.126:34933
--webserver_port=39559
--webserver_interface=127.19.52.65
--keytab_file=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517687384214-19665-0/krb5kdc/oryx_127.19.52.65.keytab
--principal=oryx/127.19.52.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.52.84:43707
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.19.52.126:34933,127.19.52.125:32823
--tserver_master_addrs=127.19.52.126:34933,127.19.52.125:32823,127.19.52.124:45647 with env {KRB5CCNAME=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517687384214-19665-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517687384214-19665-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517687384214-19665-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 06:55:24.022476 26848 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:23.994325 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44480 (local address 127.19.52.126:34933)
0812 06:55:23.994458 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0812 06:55:23.994463 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 06:55:23.994476 (+ 13us) server_negotiation.cc:407] Connection header received
0812 06:55:23.994629 (+ 153us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:55:23.994633 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:55:23.994696 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:55:23.994793 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:55:23.995362 (+ 569us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:23.996205 (+ 843us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:23.997410 (+ 1205us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:23.997767 (+ 357us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:23.997834 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:55:23.999940 (+ 2106us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:55:23.999946 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:55:23.999951 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:55:23.999982 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:55:24.006012 (+ 6030us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:55:24.006860 (+ 848us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:55:24.006864 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:55:24.006866 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:55:24.006964 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:55:24.021089 (+ 14125us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:55:24.021095 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:55:24.021097 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:55:24.021447 (+ 350us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:55:24.021551 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0812 06:55:24.022096 (+ 545us) server_negotiation.cc:299] Negotiation successful
0812 06:55:24.022270 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260812 06:55:24.022886 27642 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:23.993943 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36292 (local address 127.19.52.124:45647)
0812 06:55:23.994102 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0812 06:55:23.994107 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 06:55:23.994708 (+ 601us) server_negotiation.cc:407] Connection header received
0812 06:55:23.994830 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:55:23.994832 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:55:23.994871 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:55:23.994919 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:55:23.995471 (+ 552us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:23.995958 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:23.997394 (+ 1436us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:23.997703 (+ 309us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:23.997773 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:55:24.007004 (+ 9231us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:55:24.007010 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:55:24.007016 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:55:24.007046 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:55:24.008887 (+ 1841us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:55:24.021473 (+ 12586us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:55:24.021479 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:55:24.021483 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:55:24.021588 (+ 105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:55:24.021896 (+ 308us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:55:24.021899 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:55:24.021901 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:55:24.022121 (+ 220us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:55:24.022194 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0812 06:55:24.022562 (+ 368us) server_negotiation.cc:299] Negotiation successful
0812 06:55:24.022703 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260812 06:55:24.023617 27667 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:23.993824 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.126:34933 (local address 127.0.0.1:44480)
0812 06:55:23.994376 (+ 552us) negotiation.cc:107] Waiting for socket to connect
0812 06:55:23.994392 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0812 06:55:23.994489 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:55:23.994820 (+ 331us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:55:23.994823 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:55:23.995075 (+ 252us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:55:23.995219 (+ 144us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:55:23.995229 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:55:23.996323 (+ 1094us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:55:23.996326 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:55:23.997267 (+ 941us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:55:23.997279 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:55:23.997990 (+ 711us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:55:23.997995 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:55:23.998104 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:55:23.999299 (+ 1195us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 06:55:23.999318 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:55:23.999786 (+ 468us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 06:55:24.006154 (+ 6368us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:55:24.006158 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:55:24.006160 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:55:24.006721 (+ 561us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:55:24.007083 (+ 362us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:55:24.007086 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:55:24.007088 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:55:24.020819 (+ 13731us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:55:24.021639 (+ 820us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 06:55:24.021643 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 06:55:24.021778 (+ 135us) client_negotiation.cc:769] Sending connection context
0812 06:55:24.023200 (+ 1422us) client_negotiation.cc:240] Negotiation successful
0812 06:55:24.023428 (+ 228us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":461,"mutex_wait_us":133,"thread_start_us":132,"threads_started":1}
I20260812 06:55:24.023615 27666 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:23.994063 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.124:45647 (local address 127.0.0.1:36292)
0812 06:55:23.994612 (+ 549us) negotiation.cc:107] Waiting for socket to connect
0812 06:55:23.994624 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 06:55:23.994743 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:55:23.995047 (+ 304us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:55:23.995052 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:55:23.995246 (+ 194us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:55:23.995361 (+ 115us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:55:23.995367 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:55:23.996073 (+ 706us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:55:23.996076 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:55:23.997270 (+ 1194us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:55:23.997279 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:55:24.004825 (+ 7546us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:55:24.004857 (+ 32us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:55:24.005023 (+ 166us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:55:24.006241 (+ 1218us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 06:55:24.006254 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:55:24.006852 (+ 598us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 06:55:24.009026 (+ 2174us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:55:24.009030 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:55:24.009032 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:55:24.021296 (+ 12264us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:55:24.021700 (+ 404us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:55:24.021702 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:55:24.021704 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:55:24.021794 (+ 90us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:55:24.022262 (+ 468us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 06:55:24.022265 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 06:55:24.022365 (+ 100us) client_negotiation.cc:769] Sending connection context
0812 06:55:24.023169 (+ 804us) client_negotiation.cc:240] Negotiation successful
0812 06:55:24.023376 (+ 207us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":484,"mutex_wait_us":47,"thread_start_us":85,"threads_started":1}
I20260812 06:55:24.029470 27645 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:23.993983 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55260 (local address 127.19.52.125:32823)
0812 06:55:23.994102 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0812 06:55:23.994107 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 06:55:24.012940 (+ 18833us) server_negotiation.cc:407] Connection header received
0812 06:55:24.013069 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:55:24.013073 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:55:24.013132 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:55:24.013229 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:55:24.013876 (+ 647us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:24.014418 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:24.015676 (+ 1258us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:24.016050 (+ 374us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:24.016166 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:55:24.024561 (+ 8395us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:55:24.024566 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:55:24.024569 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:55:24.024603 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:55:24.026868 (+ 2265us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:55:24.027558 (+ 690us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:55:24.027563 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:55:24.027564 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:55:24.027670 (+ 106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:55:24.028042 (+ 372us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:55:24.028046 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:55:24.028048 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:55:24.028327 (+ 279us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:55:24.028409 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0812 06:55:24.029114 (+ 705us) server_negotiation.cc:299] Negotiation successful
0812 06:55:24.029274 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260812 06:55:24.030299 27668 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:23.994199 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.125:32823 (local address 127.0.0.1:55260)
0812 06:55:24.012830 (+ 18631us) negotiation.cc:107] Waiting for socket to connect
0812 06:55:24.012844 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 06:55:24.012977 (+ 133us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:55:24.013385 (+ 408us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:55:24.013388 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:55:24.013618 (+ 230us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:55:24.013739 (+ 121us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:55:24.013745 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:55:24.014547 (+ 802us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:55:24.014550 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:55:24.015518 (+ 968us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:55:24.015532 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:55:24.016213 (+ 681us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:55:24.016216 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:55:24.016296 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:55:24.016823 (+ 527us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 06:55:24.016834 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:55:24.024297 (+ 7463us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 06:55:24.027086 (+ 2789us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:55:24.027092 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:55:24.027094 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:55:24.027404 (+ 310us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:55:24.027805 (+ 401us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:55:24.027808 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:55:24.027810 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:55:24.027906 (+ 96us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:55:24.028756 (+ 850us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 06:55:24.028762 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 06:55:24.028877 (+ 115us) client_negotiation.cc:769] Sending connection context
0812 06:55:24.029912 (+ 1035us) client_negotiation.cc:240] Negotiation successful
0812 06:55:24.030098 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":18547,"thread_start_us":102,"threads_started":1}
Aug 12 06:55:24 dist-test-slave-65mx krb5kdc[26753](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517722, etypes {rep=17 tkt=17 ses=17}, oryx/127.19.52.126@KRBTEST.COM for oryx/127.19.52.124@KRBTEST.COM
I20260812 06:55:24.039397 27642 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:24.029436 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36304 (local address 127.19.52.124:45647)
0812 06:55:24.029580 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0812 06:55:24.029585 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 06:55:24.029601 (+ 16us) server_negotiation.cc:407] Connection header received
0812 06:55:24.029677 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:55:24.029681 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:55:24.029755 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:55:24.029834 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:55:24.030446 (+ 612us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:24.031185 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:24.032497 (+ 1312us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:24.032896 (+ 399us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:24.033005 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:55:24.035697 (+ 2692us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:55:24.035700 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:55:24.035703 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:55:24.035731 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:55:24.037316 (+ 1585us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:55:24.038016 (+ 700us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:55:24.038022 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:55:24.038024 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:55:24.038118 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:55:24.038443 (+ 325us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:55:24.038449 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:55:24.038451 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:55:24.038658 (+ 207us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:55:24.038781 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0812 06:55:24.039083 (+ 302us) server_negotiation.cc:299] Negotiation successful
0812 06:55:24.039211 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260812 06:55:24.039397 27445 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:24.028898 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.124:45647 (local address 127.0.0.1:36304)
0812 06:55:24.029184 (+ 286us) negotiation.cc:107] Waiting for socket to connect
0812 06:55:24.029201 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0812 06:55:24.029285 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:55:24.029969 (+ 684us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:55:24.029972 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:55:24.030186 (+ 214us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:55:24.030305 (+ 119us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:55:24.030317 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:55:24.031303 (+ 986us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:55:24.031309 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:55:24.032272 (+ 963us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:55:24.032285 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:55:24.033135 (+ 850us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:55:24.033143 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:55:24.033240 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:55:24.033942 (+ 702us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 06:55:24.033958 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:55:24.035401 (+ 1443us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 06:55:24.037638 (+ 2237us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:55:24.037642 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:55:24.037644 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:55:24.037894 (+ 250us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:55:24.038229 (+ 335us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:55:24.038232 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:55:24.038234 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:55:24.038339 (+ 105us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:55:24.038769 (+ 430us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 06:55:24.038772 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 06:55:24.038892 (+ 120us) client_negotiation.cc:769] Sending connection context
0812 06:55:24.039017 (+ 125us) client_negotiation.cc:240] Negotiation successful
0812 06:55:24.039205 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":202}
I20260812 06:55:24.041088 26802 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.19.52.124:45647: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "950b5d35439b4e23bd6d86c6664ddc43"
member_type: NON_VOTER
last_known_addr {
host: "127.19.52.124"
port: 45647
}
attrs {
promote: true
}
}
dest_uuid: "53e6ebc3b8ee4672983fb82b50148e26"
cas_config_opid_index: 5
I20260812 06:55:24.041740 26802 consensus_queue.cc:237] T 00000000000000000000000000000000 P 53e6ebc3b8ee4672983fb82b50148e26 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "53e6ebc3b8ee4672983fb82b50148e26" member_type: VOTER last_known_addr { host: "127.19.52.126" port: 34933 } } peers { permanent_uuid: "2b5a510eb7be4cfab2f7914ebabaa929" member_type: NON_VOTER last_known_addr { host: "127.19.52.125" port: 32823 } attrs { promote: true } } peers { permanent_uuid: "950b5d35439b4e23bd6d86c6664ddc43" member_type: NON_VOTER last_known_addr { host: "127.19.52.124" port: 45647 } attrs { promote: true } }
I20260812 06:55:24.043310 27452 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 53e6ebc3b8ee4672983fb82b50148e26 [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER 950b5d35439b4e23bd6d86c6664ddc43 (127.19.52.124) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "53e6ebc3b8ee4672983fb82b50148e26" member_type: VOTER last_known_addr { host: "127.19.52.126" port: 34933 } } peers { permanent_uuid: "2b5a510eb7be4cfab2f7914ebabaa929" member_type: NON_VOTER last_known_addr { host: "127.19.52.125" port: 32823 } attrs { promote: true } } peers { permanent_uuid: "950b5d35439b4e23bd6d86c6664ddc43" member_type: NON_VOTER last_known_addr { host: "127.19.52.124" port: 45647 } attrs { promote: true } } }
I20260812 06:55:24.045189 27452 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.19.52.124:45647
I20260812 06:55:24.045378 27452 sys_catalog.cc:455] T 00000000000000000000000000000000 P 53e6ebc3b8ee4672983fb82b50148e26 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "53e6ebc3b8ee4672983fb82b50148e26" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "53e6ebc3b8ee4672983fb82b50148e26" member_type: VOTER last_known_addr { host: "127.19.52.126" port: 34933 } } peers { permanent_uuid: "2b5a510eb7be4cfab2f7914ebabaa929" member_type: NON_VOTER last_known_addr { host: "127.19.52.125" port: 32823 } attrs { promote: true } } peers { permanent_uuid: "950b5d35439b4e23bd6d86c6664ddc43" member_type: NON_VOTER last_known_addr { host: "127.19.52.124" port: 45647 } attrs { promote: true } } }
I20260812 06:55:24.045485 27452 sys_catalog.cc:458] T 00000000000000000000000000000000 P 53e6ebc3b8ee4672983fb82b50148e26 [sys.catalog]: This master's current role is: LEADER
I20260812 06:55:24.046679 27609 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 950b5d35439b4e23bd6d86c6664ddc43 [term 0 FOLLOWER]: Advancing to term 1
I20260812 06:55:24.048322 27609 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 950b5d35439b4e23bd6d86c6664ddc43 [term 1 FOLLOWER]: Refusing update from remote peer 53e6ebc3b8ee4672983fb82b50148e26: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260812 06:55:24.049160 27453 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 53e6ebc3b8ee4672983fb82b50148e26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "950b5d35439b4e23bd6d86c6664ddc43" member_type: NON_VOTER last_known_addr { host: "127.19.52.124" port: 45647 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 06:55:24.049345 27456 master.cc:537] Master@127.19.52.124:45647 shutting down...
I20260812 06:55:24.050971 27645 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:24.043713 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55272 (local address 127.19.52.125:32823)
0812 06:55:24.043861 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0812 06:55:24.043866 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 06:55:24.044045 (+ 179us) server_negotiation.cc:407] Connection header received
0812 06:55:24.044251 (+ 206us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:55:24.044254 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:55:24.044320 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:55:24.044395 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 06:55:24.044944 (+ 549us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:24.045854 (+ 910us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:24.047645 (+ 1791us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:24.048393 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:24.048509 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:55:24.048725 (+ 216us) server_negotiation.cc:1035] Waiting for connection context
0812 06:55:24.050696 (+ 1971us) server_negotiation.cc:299] Negotiation successful
0812 06:55:24.050825 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260812 06:55:24.051788 27445 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:24.043411 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.125:32823 (local address 127.0.0.1:55272)
0812 06:55:24.043964 (+ 553us) negotiation.cc:107] Waiting for socket to connect
0812 06:55:24.043981 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0812 06:55:24.044119 (+ 138us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:55:24.044652 (+ 533us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:55:24.044665 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:55:24.044689 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 06:55:24.044788 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:55:24.044795 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:55:24.046139 (+ 1344us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:55:24.046142 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:55:24.047477 (+ 1335us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:55:24.047492 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:55:24.048925 (+ 1433us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:55:24.048931 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:55:24.049042 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:55:24.049094 (+ 52us) client_negotiation.cc:769] Sending connection context
0812 06:55:24.051501 (+ 2407us) client_negotiation.cc:240] Negotiation successful
0812 06:55:24.051629 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":475}
I20260812 06:55:24.052896 27514 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 2b5a510eb7be4cfab2f7914ebabaa929 [term 1 LEARNER]: Refusing update from remote peer 53e6ebc3b8ee4672983fb82b50148e26: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
W20260812 06:55:24.054055 26788 consensus_peers.cc:487] T 00000000000000000000000000000000 P 53e6ebc3b8ee4672983fb82b50148e26 -> Peer 950b5d35439b4e23bd6d86c6664ddc43 (127.19.52.124:45647): Couldn't send request to peer 950b5d35439b4e23bd6d86c6664ddc43. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260812 06:55:24.054059 27453 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 53e6ebc3b8ee4672983fb82b50148e26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2b5a510eb7be4cfab2f7914ebabaa929" member_type: NON_VOTER last_known_addr { host: "127.19.52.125" port: 32823 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260812 06:55:24.056638 27514 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 2b5a510eb7be4cfab2f7914ebabaa929 [term 1 LEARNER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER 950b5d35439b4e23bd6d86c6664ddc43 (127.19.52.124) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "53e6ebc3b8ee4672983fb82b50148e26" member_type: VOTER last_known_addr { host: "127.19.52.126" port: 34933 } } peers { permanent_uuid: "2b5a510eb7be4cfab2f7914ebabaa929" member_type: NON_VOTER last_known_addr { host: "127.19.52.125" port: 32823 } attrs { promote: true } } peers { permanent_uuid: "950b5d35439b4e23bd6d86c6664ddc43" member_type: NON_VOTER last_known_addr { host: "127.19.52.124" port: 45647 } attrs { promote: true } } }
I20260812 06:55:24.058784 27548 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2b5a510eb7be4cfab2f7914ebabaa929 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 53e6ebc3b8ee4672983fb82b50148e26. Latest consensus state: current_term: 1 leader_uuid: "53e6ebc3b8ee4672983fb82b50148e26" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "53e6ebc3b8ee4672983fb82b50148e26" member_type: VOTER last_known_addr { host: "127.19.52.126" port: 34933 } } peers { permanent_uuid: "2b5a510eb7be4cfab2f7914ebabaa929" member_type: NON_VOTER last_known_addr { host: "127.19.52.125" port: 32823 } attrs { promote: true } } peers { permanent_uuid: "950b5d35439b4e23bd6d86c6664ddc43" member_type: NON_VOTER last_known_addr { host: "127.19.52.124" port: 45647 } attrs { promote: true } } }
I20260812 06:55:24.058914 27548 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2b5a510eb7be4cfab2f7914ebabaa929 [sys.catalog]: This master's current role is: LEARNER
I20260812 06:55:24.059000 27548 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2b5a510eb7be4cfab2f7914ebabaa929 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "53e6ebc3b8ee4672983fb82b50148e26" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "53e6ebc3b8ee4672983fb82b50148e26" member_type: VOTER last_known_addr { host: "127.19.52.126" port: 34933 } } peers { permanent_uuid: "2b5a510eb7be4cfab2f7914ebabaa929" member_type: NON_VOTER last_known_addr { host: "127.19.52.125" port: 32823 } attrs { promote: true } } peers { permanent_uuid: "950b5d35439b4e23bd6d86c6664ddc43" member_type: NON_VOTER last_known_addr { host: "127.19.52.124" port: 45647 } attrs { promote: true } } }
I20260812 06:55:24.059059 27548 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2b5a510eb7be4cfab2f7914ebabaa929 [sys.catalog]: This master's current role is: LEARNER
I20260812 06:55:24.060168 27456 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 950b5d35439b4e23bd6d86c6664ddc43 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 06:55:24.060472 27456 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 950b5d35439b4e23bd6d86c6664ddc43 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 06:55:24.060606 27456 tablet_replica.cc:330] T 00000000000000000000000000000000 P 950b5d35439b4e23bd6d86c6664ddc43: stopping tablet replica
I20260812 06:55:24.060781 27453 raft_consensus.cc:1050] T 00000000000000000000000000000000 P 53e6ebc3b8ee4672983fb82b50148e26: attempting to promote NON_VOTER 2b5a510eb7be4cfab2f7914ebabaa929 to VOTER
I20260812 06:55:24.061306 27453 consensus_queue.cc:237] T 00000000000000000000000000000000 P 53e6ebc3b8ee4672983fb82b50148e26 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "53e6ebc3b8ee4672983fb82b50148e26" member_type: VOTER last_known_addr { host: "127.19.52.126" port: 34933 } } peers { permanent_uuid: "2b5a510eb7be4cfab2f7914ebabaa929" member_type: VOTER last_known_addr { host: "127.19.52.125" port: 32823 } attrs { promote: false } } peers { permanent_uuid: "950b5d35439b4e23bd6d86c6664ddc43" member_type: NON_VOTER last_known_addr { host: "127.19.52.124" port: 45647 } attrs { promote: true } }
I20260812 06:55:24.063038 27513 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 2b5a510eb7be4cfab2f7914ebabaa929 [term 1 LEARNER]: Refusing update from remote peer 53e6ebc3b8ee4672983fb82b50148e26: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20260812 06:55:24.063548 27450 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 53e6ebc3b8ee4672983fb82b50148e26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2b5a510eb7be4cfab2f7914ebabaa929" member_type: VOTER last_known_addr { host: "127.19.52.125" port: 32823 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
W20260812 06:55:24.063853 26788 consensus_peers.cc:487] T 00000000000000000000000000000000 P 53e6ebc3b8ee4672983fb82b50148e26 -> Peer 950b5d35439b4e23bd6d86c6664ddc43 (127.19.52.124:45647): Couldn't send request to peer 950b5d35439b4e23bd6d86c6664ddc43. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260812 06:55:24.065404 27450 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 53e6ebc3b8ee4672983fb82b50148e26 [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, 2b5a510eb7be4cfab2f7914ebabaa929 (127.19.52.125) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "53e6ebc3b8ee4672983fb82b50148e26" member_type: VOTER last_known_addr { host: "127.19.52.126" port: 34933 } } peers { permanent_uuid: "2b5a510eb7be4cfab2f7914ebabaa929" member_type: VOTER last_known_addr { host: "127.19.52.125" port: 32823 } attrs { promote: false } } peers { permanent_uuid: "950b5d35439b4e23bd6d86c6664ddc43" member_type: NON_VOTER last_known_addr { host: "127.19.52.124" port: 45647 } attrs { promote: true } } }
I20260812 06:55:24.066987 27450 sys_catalog.cc:455] T 00000000000000000000000000000000 P 53e6ebc3b8ee4672983fb82b50148e26 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "53e6ebc3b8ee4672983fb82b50148e26" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "53e6ebc3b8ee4672983fb82b50148e26" member_type: VOTER last_known_addr { host: "127.19.52.126" port: 34933 } } peers { permanent_uuid: "2b5a510eb7be4cfab2f7914ebabaa929" member_type: VOTER last_known_addr { host: "127.19.52.125" port: 32823 } attrs { promote: false } } peers { permanent_uuid: "950b5d35439b4e23bd6d86c6664ddc43" member_type: NON_VOTER last_known_addr { host: "127.19.52.124" port: 45647 } attrs { promote: true } } }
I20260812 06:55:24.067101 27450 sys_catalog.cc:458] T 00000000000000000000000000000000 P 53e6ebc3b8ee4672983fb82b50148e26 [sys.catalog]: This master's current role is: LEADER
I20260812 06:55:24.067225 27450 sys_catalog.cc:455] T 00000000000000000000000000000000 P 53e6ebc3b8ee4672983fb82b50148e26 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "53e6ebc3b8ee4672983fb82b50148e26" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "53e6ebc3b8ee4672983fb82b50148e26" member_type: VOTER last_known_addr { host: "127.19.52.126" port: 34933 } } peers { permanent_uuid: "2b5a510eb7be4cfab2f7914ebabaa929" member_type: VOTER last_known_addr { host: "127.19.52.125" port: 32823 } attrs { promote: false } } peers { permanent_uuid: "950b5d35439b4e23bd6d86c6664ddc43" member_type: NON_VOTER last_known_addr { host: "127.19.52.124" port: 45647 } attrs { promote: true } } }
I20260812 06:55:24.067291 27450 sys_catalog.cc:458] T 00000000000000000000000000000000 P 53e6ebc3b8ee4672983fb82b50148e26 [sys.catalog]: This master's current role is: LEADER
I20260812 06:55:24.067915 27513 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 2b5a510eb7be4cfab2f7914ebabaa929 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, 2b5a510eb7be4cfab2f7914ebabaa929 (127.19.52.125) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "53e6ebc3b8ee4672983fb82b50148e26" member_type: VOTER last_known_addr { host: "127.19.52.126" port: 34933 } } peers { permanent_uuid: "2b5a510eb7be4cfab2f7914ebabaa929" member_type: VOTER last_known_addr { host: "127.19.52.125" port: 32823 } attrs { promote: false } } peers { permanent_uuid: "950b5d35439b4e23bd6d86c6664ddc43" member_type: NON_VOTER last_known_addr { host: "127.19.52.124" port: 45647 } attrs { promote: true } } }
I20260812 06:55:24.070391 27674 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2b5a510eb7be4cfab2f7914ebabaa929 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "53e6ebc3b8ee4672983fb82b50148e26" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "53e6ebc3b8ee4672983fb82b50148e26" member_type: VOTER last_known_addr { host: "127.19.52.126" port: 34933 } } peers { permanent_uuid: "2b5a510eb7be4cfab2f7914ebabaa929" member_type: VOTER last_known_addr { host: "127.19.52.125" port: 32823 } attrs { promote: false } } peers { permanent_uuid: "950b5d35439b4e23bd6d86c6664ddc43" member_type: NON_VOTER last_known_addr { host: "127.19.52.124" port: 45647 } attrs { promote: true } } }
I20260812 06:55:24.070583 27674 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2b5a510eb7be4cfab2f7914ebabaa929 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 06:55:24.071063 27456 master.cc:552] Master@127.19.52.124:45647 shutdown complete.
I20260812 06:55:24.071122 27456 master_runner.cc:305] Clearing existing system tablet
I20260812 06:55:24.071383 27456 fs_manager.cc:359] Metadata directory not provided
I20260812 06:55:24.071461 27456 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:55:24.073027 27456 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260812 06:55:24.074015 27676 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 06:55:24.074291 27456 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517687384214-19665-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:55:24.074606 27456 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260812 06:55:24.074692 27456 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517687384214-19665-0/minicluster-data/master-2/data,/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517687384214-19665-0/minicluster-data/master-2/wal
uuid: "950b5d35439b4e23bd6d86c6664ddc43"
format_stamp: "Formatted at 2026-08-12 06:55:23 on dist-test-slave-65mx"
server_key: "f953733894525dc31a97d6d4b4758319"
server_key_iv: "25088ae180f6f66f48b8976c89c7ced9"
server_key_version: "encryptionkey@0"
I20260812 06:55:24.076960 27456 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P 950b5d35439b4e23bd6d86c6664ddc43: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 06:55:24.079245 27178 heartbeater.cc:503] Master 127.19.52.126:34933 was elected leader, sending a full tablet report...
I20260812 06:55:24.079711 27456 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P 950b5d35439b4e23bd6d86c6664ddc43: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260812 06:55:24.079802 27456 log.cc:1196] T 00000000000000000000000000000000 P 950b5d35439b4e23bd6d86c6664ddc43: Deleting WAL directory at /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517687384214-19665-0/minicluster-data/master-2/wal/wals/00000000000000000000000000000000
I20260812 06:55:24.080050 27456 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P 950b5d35439b4e23bd6d86c6664ddc43: Deleting consensus metadata
I20260812 06:55:24.080396 27456 master_runner.cc:315] Copying system tablet from 127.19.52.126:34933
I20260812 06:55:24.081815 27456 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P 950b5d35439b4e23bd6d86c6664ddc43: tablet copy: Beginning tablet copy session from remote peer at address 127.19.52.126:34933
I20260812 06:55:24.093406 26848 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:24.082591 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44492 (local address 127.19.52.126:34933)
0812 06:55:24.082726 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0812 06:55:24.082731 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 06:55:24.082992 (+ 261us) server_negotiation.cc:407] Connection header received
0812 06:55:24.083760 (+ 768us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:55:24.083763 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:55:24.083829 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:55:24.083933 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:55:24.085271 (+ 1338us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:24.086131 (+ 860us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:24.087332 (+ 1201us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:24.087733 (+ 401us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:24.087820 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:55:24.089449 (+ 1629us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:55:24.089454 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:55:24.089457 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:55:24.089485 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:55:24.091251 (+ 1766us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:55:24.091872 (+ 621us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:55:24.091875 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:55:24.091877 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:55:24.091968 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:55:24.092276 (+ 308us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:55:24.092279 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:55:24.092281 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:55:24.092530 (+ 249us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:55:24.092705 (+ 175us) server_negotiation.cc:1035] Waiting for connection context
0812 06:55:24.093103 (+ 398us) server_negotiation.cc:299] Negotiation successful
0812 06:55:24.093239 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260812 06:55:24.094283 27682 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:24.082482 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.126:34933 (local address 127.0.0.1:44492)
0812 06:55:24.082877 (+ 395us) negotiation.cc:107] Waiting for socket to connect
0812 06:55:24.082893 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0812 06:55:24.083170 (+ 277us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:55:24.084118 (+ 948us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:55:24.084123 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:55:24.084406 (+ 283us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:55:24.085076 (+ 670us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:55:24.085092 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:55:24.086294 (+ 1202us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:55:24.086297 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:55:24.087181 (+ 884us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:55:24.087193 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:55:24.088005 (+ 812us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:55:24.088008 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:55:24.088106 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:55:24.088814 (+ 708us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 06:55:24.088829 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:55:24.089275 (+ 446us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 06:55:24.091429 (+ 2154us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:55:24.091433 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:55:24.091435 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:55:24.091730 (+ 295us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:55:24.092080 (+ 350us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:55:24.092083 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:55:24.092085 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:55:24.092174 (+ 89us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:55:24.092802 (+ 628us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 06:55:24.092804 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 06:55:24.092914 (+ 110us) client_negotiation.cc:769] Sending connection context
0812 06:55:24.093923 (+ 1009us) client_negotiation.cc:240] Negotiation successful
0812 06:55:24.094086 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":306,"thread_start_us":107,"threads_started":1}
I20260812 06:55:24.094996 26825 tablet_copy_service.cc:140] P 53e6ebc3b8ee4672983fb82b50148e26: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer 950b5d35439b4e23bd6d86c6664ddc43 ({username='oryx', principal='oryx/127.19.52.124@KRBTEST.COM'} at 127.0.0.1:44492)
I20260812 06:55:24.095096 26825 tablet_copy_service.cc:161] P 53e6ebc3b8ee4672983fb82b50148e26: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer 950b5d35439b4e23bd6d86c6664ddc43 at {username='oryx', principal='oryx/127.19.52.124@KRBTEST.COM'} at 127.0.0.1:44492: session id = 950b5d35439b4e23bd6d86c6664ddc43-00000000000000000000000000000000
I20260812 06:55:24.096328 26825 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P 53e6ebc3b8ee4672983fb82b50148e26: Tablet Copy: opened 0 blocks and 1 log segments
I20260812 06:55:24.097792 27456 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260812 06:55:24.100342 27456 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P 950b5d35439b4e23bd6d86c6664ddc43: tablet copy: Starting download of 0 data blocks...
I20260812 06:55:24.100574 27456 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P 950b5d35439b4e23bd6d86c6664ddc43: tablet copy: Starting download of 1 WAL segments...
I20260812 06:55:24.102317 27456 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P 950b5d35439b4e23bd6d86c6664ddc43: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260812 06:55:24.104445 26825 tablet_copy_service.cc:342] P 53e6ebc3b8ee4672983fb82b50148e26: Request end of tablet copy session 950b5d35439b4e23bd6d86c6664ddc43-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.19.52.124@KRBTEST.COM'} at 127.0.0.1:44492
I20260812 06:55:24.104609 26825 tablet_copy_service.cc:433] P 53e6ebc3b8ee4672983fb82b50148e26: ending tablet copy session 950b5d35439b4e23bd6d86c6664ddc43-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer 950b5d35439b4e23bd6d86c6664ddc43
I20260812 06:55:24.113621 27456 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 06:55:24.115868 27684 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:55:24.115921 27683 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:55:24.116159 27687 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:55:24.120427 27456 server_base.cc:1034] running on GCE node
I20260812 06:55:24.120870 27456 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:55:24.121275 27456 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:55:24.122539 27456 hybrid_clock.cc:648] HybridClock initialized: now 1786517724122521 us; error 41 us; skew 500 ppm
Aug 12 06:55:24 dist-test-slave-65mx krb5kdc[26753](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517724, etypes {rep=17 tkt=17 ses=17}, oryx/127.19.52.124@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 06:55:24.124571 27456 init.cc:377] Logged in from keytab as oryx/127.19.52.124@KRBTEST.COM (short username oryx)
I20260812 06:55:24.125207 27456 webserver.cc:460] Webserver started at http://127.19.52.124:34203/ using document root <none> and password file <none>
I20260812 06:55:24.125429 27456 fs_manager.cc:359] Metadata directory not provided
I20260812 06:55:24.125487 27456 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:55:24.127305 27456 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260812 06:55:24.128630 27695 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 06:55:24.128968 27456 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 06:55:24.129128 27456 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517687384214-19665-0/minicluster-data/master-2/data,/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517687384214-19665-0/minicluster-data/master-2/wal
uuid: "950b5d35439b4e23bd6d86c6664ddc43"
format_stamp: "Formatted at 2026-08-12 06:55:23 on dist-test-slave-65mx"
server_key: "f953733894525dc31a97d6d4b4758319"
server_key_iv: "25088ae180f6f66f48b8976c89c7ced9"
server_key_version: "encryptionkey@0"
I20260812 06:55:24.129302 27456 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517687384214-19665-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517687384214-19665-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517687384214-19665-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:55:24.151018 27456 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 06:55:24.153894 27456 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:55:24.159960 27456 rpc_server.cc:307] RPC server started. Bound to: 127.19.52.124:45647
I20260812 06:55:24.159993 27752 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.52.124:45647 every 8 connection(s)
I20260812 06:55:24.160964 27456 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517687384214-19665-0/minicluster-data/master-2/data/info.pb
I20260812 06:55:24.163089 27753 sys_catalog.cc:263] Verifying existing consensus state
I20260812 06:55:24.164383 27753 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 950b5d35439b4e23bd6d86c6664ddc43: Bootstrap starting.
I20260812 06:55:24.177131 27753 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 950b5d35439b4e23bd6d86c6664ddc43: Bootstrap replayed 1/1 log segments. Stats: ops{read=7 overwritten=0 applied=7 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260812 06:55:24.177541 27753 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 950b5d35439b4e23bd6d86c6664ddc43: Bootstrap complete.
I20260812 06:55:24.178229 27753 raft_consensus.cc:348] T 00000000000000000000000000000000 P 950b5d35439b4e23bd6d86c6664ddc43 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "53e6ebc3b8ee4672983fb82b50148e26" member_type: VOTER last_known_addr { host: "127.19.52.126" port: 34933 } } peers { permanent_uuid: "2b5a510eb7be4cfab2f7914ebabaa929" member_type: VOTER last_known_addr { host: "127.19.52.125" port: 32823 } attrs { promote: false } } peers { permanent_uuid: "950b5d35439b4e23bd6d86c6664ddc43" member_type: NON_VOTER last_known_addr { host: "127.19.52.124" port: 45647 } attrs { promote: true } }
I20260812 06:55:24.178395 27753 raft_consensus.cc:729] T 00000000000000000000000000000000 P 950b5d35439b4e23bd6d86c6664ddc43 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 950b5d35439b4e23bd6d86c6664ddc43, State: Initialized, Role: LEARNER
I20260812 06:55:24.178508 27753 consensus_queue.cc:260] T 00000000000000000000000000000000 P 950b5d35439b4e23bd6d86c6664ddc43 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 7, Last appended: 1.7, Last appended by leader: 7, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "53e6ebc3b8ee4672983fb82b50148e26" member_type: VOTER last_known_addr { host: "127.19.52.126" port: 34933 } } peers { permanent_uuid: "2b5a510eb7be4cfab2f7914ebabaa929" member_type: VOTER last_known_addr { host: "127.19.52.125" port: 32823 } attrs { promote: false } } peers { permanent_uuid: "950b5d35439b4e23bd6d86c6664ddc43" member_type: NON_VOTER last_known_addr { host: "127.19.52.124" port: 45647 } attrs { promote: true } }
I20260812 06:55:24.178922 27756 sys_catalog.cc:455] T 00000000000000000000000000000000 P 950b5d35439b4e23bd6d86c6664ddc43 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "53e6ebc3b8ee4672983fb82b50148e26" member_type: VOTER last_known_addr { host: "127.19.52.126" port: 34933 } } peers { permanent_uuid: "2b5a510eb7be4cfab2f7914ebabaa929" member_type: VOTER last_known_addr { host: "127.19.52.125" port: 32823 } attrs { promote: false } } peers { permanent_uuid: "950b5d35439b4e23bd6d86c6664ddc43" member_type: NON_VOTER last_known_addr { host: "127.19.52.124" port: 45647 } attrs { promote: true } } }
I20260812 06:55:24.179087 27753 sys_catalog.cc:564] T 00000000000000000000000000000000 P 950b5d35439b4e23bd6d86c6664ddc43 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 06:55:24.179136 27756 sys_catalog.cc:458] T 00000000000000000000000000000000 P 950b5d35439b4e23bd6d86c6664ddc43 [sys.catalog]: This master's current role is: LEARNER
I20260812 06:55:24.183112 27769 catalog_manager.cc:1259] Loaded cluster ID: 020dffafdb2c4d38b19668de7f5df860
I20260812 06:55:24.183286 27769 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 950b5d35439b4e23bd6d86c6664ddc43: loading cluster ID for follower catalog manager: success
I20260812 06:55:24.185948 27769 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 950b5d35439b4e23bd6d86c6664ddc43: acquiring CA information for follower catalog manager: success
I20260812 06:55:24.187736 27769 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 950b5d35439b4e23bd6d86c6664ddc43: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260812 06:55:24.191304 27669 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:55:24.191622 27669 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:55:24.191673 27669 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 06:55:24.196698 27669 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 06:55:24.196777 27669 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 06:55:24.196872 27669 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.52.65
I20260812 06:55:24.202919 27669 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.52.84:43707
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517687384214-19665-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517687384214-19665-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517687384214-19665-0/krb5kdc/oryx_127.19.52.65.keytab
--principal=oryx/127.19.52.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.52.65:41417
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517687384214-19665-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.19.52.65
--webserver_port=39559
--tserver_master_addrs=127.19.52.126:34933,127.19.52.125:32823,127.19.52.124:45647
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27669
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.52.65
--log_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517687384214-19665-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 06:15:54 UTC on ccycloud.quasar-rnfzce.root.comops.site
build id 4581989
I20260812 06:55:24.204320 27669 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:55:24.205693 27669 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 06:55:24.216472 27775 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:55:24.216477 27774 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:55:24.216470 27777 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:55:24.216933 27669 server_base.cc:1034] running on GCE node
I20260812 06:55:24.217422 27669 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:55:24.218150 27669 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:55:24.219350 27669 hybrid_clock.cc:648] HybridClock initialized: now 1786517724219311 us; error 51 us; skew 500 ppm
Aug 12 06:55:24 dist-test-slave-65mx krb5kdc[26753](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517724, etypes {rep=17 tkt=17 ses=17}, oryx/127.19.52.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 06:55:24.222594 27669 init.cc:377] Logged in from keytab as oryx/127.19.52.65@KRBTEST.COM (short username oryx)
I20260812 06:55:24.223560 27669 webserver.cc:460] Webserver started at http://127.19.52.65:39559/ using document root <none> and password file <none>
I20260812 06:55:24.224123 27669 fs_manager.cc:359] Metadata directory not provided
I20260812 06:55:24.224197 27669 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:55:24.228426 27669 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.001s
I20260812 06:55:24.231251 27787 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 06:55:24.232996 27669 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 06:55:24.233189 27669 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517687384214-19665-0/minicluster-data/ts-0/data,/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517687384214-19665-0/minicluster-data/ts-0/wal
uuid: "49e259bbf4094665b4e899332664889c"
format_stamp: "Formatted at 2026-08-12 06:55:22 on dist-test-slave-65mx"
server_key: "7c90dd6a5388828c4ca99ec509ea42f5"
server_key_iv: "da23e9e61d2d1349b6ecc96ac85fa6a1"
server_key_version: "encryptionkey@0"
I20260812 06:55:24.233877 27669 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517687384214-19665-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517687384214-19665-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517687384214-19665-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:55:24.263628 27669 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 06:55:24.267774 27669 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 06:55:24.268039 27669 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:55:24.268762 27669 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 06:55:24.269738 27669 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 06:55:24.269816 27669 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 06:55:24.269892 27669 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 06:55:24.269951 27669 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 06:55:24.280661 27669 rpc_server.cc:307] RPC server started. Bound to: 127.19.52.65:41417
I20260812 06:55:24.280714 27922 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.52.65:41417 every 8 connection(s)
I20260812 06:55:24.281893 27669 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517687384214-19665-0/minicluster-data/ts-0/data/info.pb
I20260812 06:55:24.285620 19665 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu as pid 27669
I20260812 06:55:24.285861 19665 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu with pid 27025
Aug 12 06:55:24 dist-test-slave-65mx krb5kdc[26753](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517724, etypes {rep=17 tkt=17 ses=17}, oryx/127.19.52.65@KRBTEST.COM for oryx/127.19.52.125@KRBTEST.COM
I20260812 06:55:24.296972 19665 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu
/tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517687384214-19665-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517687384214-19665-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517687384214-19665-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517687384214-19665-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.52.66:34045
--local_ip_for_outbound_sockets=127.19.52.66
--tserver_master_addrs=127.19.52.126:34933
--webserver_port=35657
--webserver_interface=127.19.52.66
--keytab_file=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517687384214-19665-0/krb5kdc/oryx_127.19.52.66.keytab
--principal=oryx/127.19.52.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.52.84:43707
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.19.52.126:34933,127.19.52.125:32823
--tserver_master_addrs=127.19.52.126:34933,127.19.52.125:32823,127.19.52.124:45647 with env {KRB5CCNAME=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517687384214-19665-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517687384214-19665-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517687384214-19665-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 12 06:55:24 dist-test-slave-65mx krb5kdc[26753](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517724, etypes {rep=17 tkt=17 ses=17}, oryx/127.19.52.65@KRBTEST.COM for oryx/127.19.52.126@KRBTEST.COM
Aug 12 06:55:24 dist-test-slave-65mx krb5kdc[26753](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517724, etypes {rep=17 tkt=17 ses=17}, oryx/127.19.52.65@KRBTEST.COM for oryx/127.19.52.124@KRBTEST.COM
I20260812 06:55:24.302598 27645 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:24.285790 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.52.65:38857 (local address 127.19.52.125:32823)
0812 06:55:24.285918 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0812 06:55:24.285922 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 06:55:24.285938 (+ 16us) server_negotiation.cc:407] Connection header received
0812 06:55:24.287437 (+ 1499us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:55:24.287441 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:55:24.287510 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:55:24.287686 (+ 176us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:55:24.289634 (+ 1948us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:24.290649 (+ 1015us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:24.293330 (+ 2681us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:24.293775 (+ 445us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:24.293879 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:55:24.297790 (+ 3911us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:55:24.297797 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:55:24.297802 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:55:24.297842 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:55:24.299601 (+ 1759us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:55:24.300736 (+ 1135us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:55:24.300740 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:55:24.300742 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:55:24.300834 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:55:24.301257 (+ 423us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:55:24.301260 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:55:24.301262 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:55:24.301510 (+ 248us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:55:24.301600 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0812 06:55:24.302292 (+ 692us) server_negotiation.cc:299] Negotiation successful
0812 06:55:24.302429 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260812 06:55:24.304934 26848 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:24.283808 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.52.65:51241 (local address 127.19.52.126:34933)
0812 06:55:24.283985 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0812 06:55:24.283991 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 06:55:24.285773 (+ 1782us) server_negotiation.cc:407] Connection header received
0812 06:55:24.287610 (+ 1837us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:55:24.287616 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:55:24.287677 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:55:24.287728 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:55:24.290023 (+ 2295us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:24.290812 (+ 789us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:24.293308 (+ 2496us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:24.293676 (+ 368us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:24.293765 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:55:24.299284 (+ 5519us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:55:24.299288 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:55:24.299290 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:55:24.299348 (+ 58us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:55:24.301259 (+ 1911us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:55:24.302011 (+ 752us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:55:24.302015 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:55:24.302017 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:55:24.302145 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:55:24.302526 (+ 381us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:55:24.302529 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:55:24.302534 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:55:24.302820 (+ 286us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:55:24.302938 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0812 06:55:24.304430 (+ 1492us) server_negotiation.cc:299] Negotiation successful
0812 06:55:24.304655 (+ 225us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260812 06:55:24.304255 27931 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:24.284298 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.126:34933 (local address 127.19.52.65:51241)
0812 06:55:24.285581 (+ 1283us) negotiation.cc:107] Waiting for socket to connect
0812 06:55:24.285628 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0812 06:55:24.287458 (+ 1830us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:55:24.288015 (+ 557us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:55:24.288024 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:55:24.289124 (+ 1100us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:55:24.289882 (+ 758us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:55:24.289887 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:55:24.290963 (+ 1076us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:55:24.290966 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:55:24.293138 (+ 2172us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:55:24.293153 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:55:24.293978 (+ 825us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:55:24.293983 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:55:24.294287 (+ 304us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:55:24.295234 (+ 947us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 06:55:24.295272 (+ 38us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:55:24.299053 (+ 3781us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 06:55:24.301510 (+ 2457us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:55:24.301517 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:55:24.301520 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:55:24.301836 (+ 316us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:55:24.302281 (+ 445us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:55:24.302287 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:55:24.302291 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:55:24.302402 (+ 111us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:55:24.303297 (+ 895us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 06:55:24.303304 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 06:55:24.303433 (+ 129us) client_negotiation.cc:769] Sending connection context
0812 06:55:24.303631 (+ 198us) client_negotiation.cc:240] Negotiation successful
0812 06:55:24.303826 (+ 195us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1199,"mutex_wait_us":1191,"spinlock_wait_cycles":11904,"thread_start_us":156,"threads_started":1}
I20260812 06:55:24.304627 27928 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:24.284298 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.125:32823 (local address 127.19.52.65:38857)
0812 06:55:24.285141 (+ 843us) negotiation.cc:107] Waiting for socket to connect
0812 06:55:24.285772 (+ 631us) client_negotiation.cc:174] Beginning negotiation
0812 06:55:24.287149 (+ 1377us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:55:24.288015 (+ 866us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:55:24.288024 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:55:24.288671 (+ 647us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:55:24.289433 (+ 762us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:55:24.289461 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:55:24.290861 (+ 1400us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:55:24.290869 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:55:24.293156 (+ 2287us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:55:24.293172 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:55:24.294081 (+ 909us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:55:24.294087 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:55:24.294343 (+ 256us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:55:24.295318 (+ 975us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 06:55:24.295338 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:55:24.297531 (+ 2193us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 06:55:24.300195 (+ 2664us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:55:24.300201 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:55:24.300218 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:55:24.300608 (+ 390us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:55:24.300957 (+ 349us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:55:24.300960 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:55:24.300962 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:55:24.301142 (+ 180us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:55:24.301675 (+ 533us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 06:55:24.301681 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 06:55:24.301987 (+ 306us) client_negotiation.cc:769] Sending connection context
0812 06:55:24.302827 (+ 840us) client_negotiation.cc:240] Negotiation successful
0812 06:55:24.303200 (+ 373us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":568,"mutex_wait_us":337,"thread_start_us":783,"threads_started":1}
I20260812 06:55:24.306823 27925 heartbeater.cc:347] Connected to a master server at 127.19.52.125:32823
I20260812 06:55:24.307102 27925 heartbeater.cc:464] Registering TS with master...
I20260812 06:55:24.307866 27925 heartbeater.cc:511] Master 127.19.52.125:32823 requested a full tablet report, sending...
I20260812 06:55:24.306823 27924 heartbeater.cc:347] Connected to a master server at 127.19.52.126:34933
I20260812 06:55:24.308481 27924 heartbeater.cc:464] Registering TS with master...
I20260812 06:55:24.308710 27924 heartbeater.cc:511] Master 127.19.52.126:34933 requested a full tablet report, sending...
I20260812 06:55:24.309604 27503 ts_manager.cc:194] Registered new tserver with Master: 49e259bbf4094665b4e899332664889c (127.19.52.65:41417)
I20260812 06:55:24.309751 26802 ts_manager.cc:194] Re-registered known tserver with Master: 49e259bbf4094665b4e899332664889c (127.19.52.65:41417)
I20260812 06:55:24.310310 27929 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:24.284384 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.124:45647 (local address 127.19.52.65:43491)
0812 06:55:24.288409 (+ 4025us) negotiation.cc:107] Waiting for socket to connect
0812 06:55:24.288425 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0812 06:55:24.288528 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:55:24.289461 (+ 933us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:55:24.289463 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:55:24.289655 (+ 192us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:55:24.289749 (+ 94us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:55:24.289754 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:55:24.291135 (+ 1381us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:55:24.291138 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:55:24.293147 (+ 2009us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:55:24.293153 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:55:24.293978 (+ 825us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:55:24.293983 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:55:24.294313 (+ 330us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:55:24.295234 (+ 921us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 06:55:24.295257 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:55:24.298843 (+ 3586us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 06:55:24.307834 (+ 8991us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:55:24.307840 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:55:24.307845 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:55:24.308219 (+ 374us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:55:24.308717 (+ 498us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:55:24.308724 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:55:24.308728 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:55:24.308849 (+ 121us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:55:24.309494 (+ 645us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 06:55:24.309500 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 06:55:24.309641 (+ 141us) client_negotiation.cc:769] Sending connection context
0812 06:55:24.309763 (+ 122us) client_negotiation.cc:240] Negotiation successful
0812 06:55:24.309978 (+ 215us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3928,"mutex_wait_us":910,"spinlock_wait_cycles":68096,"thread_start_us":155,"threads_started":1}
I20260812 06:55:24.310554 26802 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.19.52.65@KRBTEST.COM'} at 127.19.52.65:51241
I20260812 06:55:24.311074 27927 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:24.283903 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.52.65:43491 (local address 127.19.52.124:45647)
0812 06:55:24.284154 (+ 251us) server_negotiation.cc:206] Beginning negotiation
0812 06:55:24.284159 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 06:55:24.288650 (+ 4491us) server_negotiation.cc:407] Connection header received
0812 06:55:24.288706 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:55:24.288709 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:55:24.288763 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:55:24.288844 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:55:24.289887 (+ 1043us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:24.291005 (+ 1118us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:24.293308 (+ 2303us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:24.293678 (+ 370us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:24.293777 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:55:24.299832 (+ 6055us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:55:24.299835 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:55:24.299838 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:55:24.299869 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:55:24.307651 (+ 7782us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:55:24.308388 (+ 737us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:55:24.308395 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:55:24.308399 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:55:24.308520 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:55:24.308983 (+ 463us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:55:24.308989 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:55:24.308993 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:55:24.309313 (+ 320us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:55:24.309436 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0812 06:55:24.310604 (+ 1168us) server_negotiation.cc:299] Negotiation successful
0812 06:55:24.310790 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":183,"thread_start_us":84,"threads_started":1}
I20260812 06:55:24.312004 27923 heartbeater.cc:347] Connected to a master server at 127.19.52.124:45647
I20260812 06:55:24.312124 27923 heartbeater.cc:464] Registering TS with master...
I20260812 06:55:24.312498 27923 heartbeater.cc:511] Master 127.19.52.124:45647 requested a full tablet report, sending...
I20260812 06:55:24.314033 27712 ts_manager.cc:194] Registered new tserver with Master: 49e259bbf4094665b4e899332664889c (127.19.52.65:41417)
I20260812 06:55:24.322863 27333 heartbeater.cc:503] Master 127.19.52.126:34933 was elected leader, sending a full tablet report...
W20260812 06:55:24.443760 27932 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:55:24.444065 27932 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:55:24.444111 27932 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 06:55:24.448348 27932 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 06:55:24.448427 27932 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 06:55:24.448527 27932 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.52.66
I20260812 06:55:24.454056 27932 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.52.84:43707
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517687384214-19665-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517687384214-19665-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517687384214-19665-0/krb5kdc/oryx_127.19.52.66.keytab
--principal=oryx/127.19.52.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.52.66:34045
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517687384214-19665-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.19.52.66
--webserver_port=35657
--tserver_master_addrs=127.19.52.126:34933,127.19.52.125:32823,127.19.52.124:45647
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27932
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.52.66
--log_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517687384214-19665-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 06:15:54 UTC on ccycloud.quasar-rnfzce.root.comops.site
build id 4581989
I20260812 06:55:24.455277 27932 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:55:24.456826 27932 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 06:55:24.469983 27940 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:55:24.470187 27932 server_base.cc:1034] running on GCE node
W20260812 06:55:24.470196 27938 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:55:24.470079 27937 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:55:24.470916 27932 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:55:24.471699 27932 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:55:24.472904 27932 hybrid_clock.cc:648] HybridClock initialized: now 1786517724472885 us; error 35 us; skew 500 ppm
Aug 12 06:55:24 dist-test-slave-65mx krb5kdc[26753](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517724, etypes {rep=17 tkt=17 ses=17}, oryx/127.19.52.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 06:55:24.475970 27932 init.cc:377] Logged in from keytab as oryx/127.19.52.66@KRBTEST.COM (short username oryx)
I20260812 06:55:24.477185 27932 webserver.cc:460] Webserver started at http://127.19.52.66:35657/ using document root <none> and password file <none>
I20260812 06:55:24.477826 27932 fs_manager.cc:359] Metadata directory not provided
I20260812 06:55:24.477958 27932 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:55:24.482690 27932 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260812 06:55:24.485381 27948 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 06:55:24.487104 27932 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 06:55:24.487301 27932 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517687384214-19665-0/minicluster-data/ts-1/data,/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517687384214-19665-0/minicluster-data/ts-1/wal
uuid: "2e97338a875f4bada5bf4e6301558f78"
format_stamp: "Formatted at 2026-08-12 06:55:23 on dist-test-slave-65mx"
server_key: "8725a83480edc9ec5c00367522d80e77"
server_key_iv: "ecde9130367d813eeaefa9f7eaef66fc"
server_key_version: "encryptionkey@0"
I20260812 06:55:24.487967 27932 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517687384214-19665-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517687384214-19665-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517687384214-19665-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:55:24.503731 27932 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 06:55:24.508086 27932 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 06:55:24.508296 27932 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:55:24.509033 27932 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 06:55:24.510143 27932 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 06:55:24.510203 27932 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 06:55:24.510257 27932 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 06:55:24.510288 27932 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 06:55:24.521584 27932 rpc_server.cc:307] RPC server started. Bound to: 127.19.52.66:34045
I20260812 06:55:24.521656 28084 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.52.66:34045 every 8 connection(s)
I20260812 06:55:24.522781 27932 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517687384214-19665-0/minicluster-data/ts-1/data/info.pb
I20260812 06:55:24.530141 19665 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu as pid 27932
I20260812 06:55:24.530362 19665 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu with pid 27183
Aug 12 06:55:24 dist-test-slave-65mx krb5kdc[26753](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517724, etypes {rep=17 tkt=17 ses=17}, oryx/127.19.52.66@KRBTEST.COM for oryx/127.19.52.124@KRBTEST.COM
I20260812 06:55:24.543540 27927 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:24.524886 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.52.66:41145 (local address 127.19.52.124:45647)
0812 06:55:24.525168 (+ 282us) server_negotiation.cc:206] Beginning negotiation
0812 06:55:24.525173 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 06:55:24.526785 (+ 1612us) server_negotiation.cc:407] Connection header received
0812 06:55:24.527104 (+ 319us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:55:24.527108 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:55:24.527162 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:55:24.527217 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:55:24.528517 (+ 1300us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:24.529151 (+ 634us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:24.534458 (+ 5307us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:24.534874 (+ 416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:24.534973 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:55:24.538747 (+ 3774us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:55:24.538754 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:55:24.538760 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:55:24.538796 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:55:24.540760 (+ 1964us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:55:24.541508 (+ 748us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:55:24.541515 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:55:24.541520 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:55:24.541616 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:55:24.542051 (+ 435us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:55:24.542057 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:55:24.542062 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:55:24.542316 (+ 254us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:55:24.542400 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0812 06:55:24.543124 (+ 724us) server_negotiation.cc:299] Negotiation successful
0812 06:55:24.543286 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":182}
I20260812 06:55:24.546015 28091 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:24.525361 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.124:45647 (local address 127.19.52.66:41145)
0812 06:55:24.526688 (+ 1327us) negotiation.cc:107] Waiting for socket to connect
0812 06:55:24.526698 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0812 06:55:24.526901 (+ 203us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:55:24.527450 (+ 549us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:55:24.527457 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:55:24.527966 (+ 509us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:55:24.528357 (+ 391us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:55:24.528363 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:55:24.529399 (+ 1036us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:55:24.529402 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:55:24.534254 (+ 4852us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:55:24.534276 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:55:24.535237 (+ 961us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:55:24.535244 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:55:24.535514 (+ 270us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:55:24.536282 (+ 768us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 06:55:24.536310 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:55:24.538497 (+ 2187us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 06:55:24.540930 (+ 2433us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:55:24.540940 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:55:24.540961 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:55:24.541381 (+ 420us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:55:24.541737 (+ 356us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:55:24.541743 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:55:24.541748 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:55:24.541935 (+ 187us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:55:24.542472 (+ 537us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 06:55:24.542482 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 06:55:24.542831 (+ 349us) client_negotiation.cc:769] Sending connection context
0812 06:55:24.544068 (+ 1237us) client_negotiation.cc:240] Negotiation successful
0812 06:55:24.544721 (+ 653us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1260,"mutex_wait_us":339,"spinlock_wait_cycles":2816,"thread_start_us":146,"threads_started":1}
Aug 12 06:55:24 dist-test-slave-65mx krb5kdc[26753](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517724, etypes {rep=17 tkt=17 ses=17}, oryx/127.19.52.66@KRBTEST.COM for oryx/127.19.52.126@KRBTEST.COM
I20260812 06:55:24.548430 28085 heartbeater.cc:347] Connected to a master server at 127.19.52.124:45647
I20260812 06:55:24.548776 28085 heartbeater.cc:464] Registering TS with master...
I20260812 06:55:24.549504 28085 heartbeater.cc:511] Master 127.19.52.124:45647 requested a full tablet report, sending...
Aug 12 06:55:24 dist-test-slave-65mx krb5kdc[26753](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517724, etypes {rep=17 tkt=17 ses=17}, oryx/127.19.52.66@KRBTEST.COM for oryx/127.19.52.125@KRBTEST.COM
I20260812 06:55:24.551772 27712 ts_manager.cc:194] Registered new tserver with Master: 2e97338a875f4bada5bf4e6301558f78 (127.19.52.66:34045)
I20260812 06:55:24.555089 27645 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:24.525065 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.52.66:33633 (local address 127.19.52.125:32823)
0812 06:55:24.525227 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0812 06:55:24.525233 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 06:55:24.526554 (+ 1321us) server_negotiation.cc:407] Connection header received
0812 06:55:24.527106 (+ 552us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:55:24.527109 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:55:24.527162 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:55:24.527217 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:55:24.528537 (+ 1320us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:24.529393 (+ 856us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:24.544596 (+ 15203us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:24.545206 (+ 610us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:24.545334 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:55:24.550423 (+ 5089us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:55:24.550431 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:55:24.550438 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:55:24.550484 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:55:24.552713 (+ 2229us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:55:24.553275 (+ 562us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:55:24.553280 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:55:24.553282 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:55:24.553392 (+ 110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:55:24.553898 (+ 506us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:55:24.553902 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:55:24.553904 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:55:24.554200 (+ 296us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:55:24.554306 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0812 06:55:24.554771 (+ 465us) server_negotiation.cc:299] Negotiation successful
0812 06:55:24.554915 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260812 06:55:24.555119 26848 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:24.525018 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.52.66:54215 (local address 127.19.52.126:34933)
0812 06:55:24.525265 (+ 247us) server_negotiation.cc:206] Beginning negotiation
0812 06:55:24.525269 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 06:55:24.526085 (+ 816us) server_negotiation.cc:407] Connection header received
0812 06:55:24.527097 (+ 1012us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:55:24.527100 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:55:24.527161 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:55:24.527273 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:55:24.528495 (+ 1222us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:24.529363 (+ 868us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:24.545195 (+ 15832us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:24.545603 (+ 408us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:24.545704 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:55:24.550733 (+ 5029us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:55:24.550738 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:55:24.550741 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:55:24.550775 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:55:24.553186 (+ 2411us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:55:24.553724 (+ 538us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:55:24.553729 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:55:24.553731 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:55:24.553833 (+ 102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:55:24.554151 (+ 318us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:55:24.554155 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:55:24.554156 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:55:24.554363 (+ 207us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:55:24.554494 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0812 06:55:24.554844 (+ 350us) server_negotiation.cc:299] Negotiation successful
0812 06:55:24.554982 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":174}
I20260812 06:55:24.555239 28090 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:24.525361 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.125:32823 (local address 127.19.52.66:33633)
0812 06:55:24.526440 (+ 1079us) negotiation.cc:107] Waiting for socket to connect
0812 06:55:24.526452 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 06:55:24.526901 (+ 449us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:55:24.527450 (+ 549us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:55:24.527498 (+ 48us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:55:24.527963 (+ 465us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:55:24.528355 (+ 392us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:55:24.528363 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:55:24.529556 (+ 1193us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:55:24.529558 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:55:24.544336 (+ 14778us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:55:24.544356 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:55:24.545583 (+ 1227us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:55:24.545591 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:55:24.545769 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:55:24.547220 (+ 1451us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 06:55:24.547245 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:55:24.550167 (+ 2922us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 06:55:24.552862 (+ 2695us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:55:24.552868 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:55:24.552873 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:55:24.553170 (+ 297us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:55:24.553612 (+ 442us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:55:24.553618 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:55:24.553624 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:55:24.553759 (+ 135us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:55:24.554399 (+ 640us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 06:55:24.554405 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 06:55:24.554549 (+ 144us) client_negotiation.cc:769] Sending connection context
0812 06:55:24.554682 (+ 133us) client_negotiation.cc:240] Negotiation successful
0812 06:55:24.554888 (+ 206us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1002,"mutex_wait_us":139,"spinlock_wait_cycles":55808,"thread_start_us":158,"threads_started":1}
I20260812 06:55:24.555902 28089 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:24.525361 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.126:34933 (local address 127.19.52.66:54215)
0812 06:55:24.525916 (+ 555us) negotiation.cc:107] Waiting for socket to connect
0812 06:55:24.525970 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0812 06:55:24.526886 (+ 916us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:55:24.527449 (+ 563us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:55:24.527455 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:55:24.527855 (+ 400us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:55:24.528342 (+ 487us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:55:24.528355 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:55:24.529558 (+ 1203us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:55:24.529561 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:55:24.544991 (+ 15430us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:55:24.545003 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:55:24.545914 (+ 911us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:55:24.545918 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:55:24.546020 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:55:24.547019 (+ 999us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 06:55:24.547044 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:55:24.550504 (+ 3460us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 06:55:24.553330 (+ 2826us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:55:24.553336 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:55:24.553341 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:55:24.553607 (+ 266us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:55:24.553940 (+ 333us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:55:24.553945 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:55:24.553947 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:55:24.554046 (+ 99us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:55:24.554502 (+ 456us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 06:55:24.554509 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 06:55:24.554611 (+ 102us) client_negotiation.cc:769] Sending connection context
0812 06:55:24.555362 (+ 751us) client_negotiation.cc:240] Negotiation successful
0812 06:55:24.555694 (+ 332us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":307,"mutex_wait_us":56,"thread_start_us":156,"threads_started":1}
I20260812 06:55:24.556015 28087 heartbeater.cc:347] Connected to a master server at 127.19.52.125:32823
I20260812 06:55:24.556195 28087 heartbeater.cc:464] Registering TS with master...
I20260812 06:55:24.556707 28087 heartbeater.cc:511] Master 127.19.52.125:32823 requested a full tablet report, sending...
I20260812 06:55:24.557804 27503 ts_manager.cc:194] Registered new tserver with Master: 2e97338a875f4bada5bf4e6301558f78 (127.19.52.66:34045)
I20260812 06:55:24.558199 28086 heartbeater.cc:347] Connected to a master server at 127.19.52.126:34933
I20260812 06:55:24.558307 28086 heartbeater.cc:464] Registering TS with master...
I20260812 06:55:24.558645 28086 heartbeater.cc:511] Master 127.19.52.126:34933 requested a full tablet report, sending...
I20260812 06:55:24.559480 26802 ts_manager.cc:194] Re-registered known tserver with Master: 2e97338a875f4bada5bf4e6301558f78 (127.19.52.66:34045)
I20260812 06:55:24.559998 19665 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu
/tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517687384214-19665-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517687384214-19665-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517687384214-19665-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517687384214-19665-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.52.67:45705
--local_ip_for_outbound_sockets=127.19.52.67
--tserver_master_addrs=127.19.52.126:34933
--webserver_port=38493
--webserver_interface=127.19.52.67
--keytab_file=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517687384214-19665-0/krb5kdc/oryx_127.19.52.67.keytab
--principal=oryx/127.19.52.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.52.84:43707
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.19.52.126:34933,127.19.52.125:32823
--tserver_master_addrs=127.19.52.126:34933,127.19.52.125:32823,127.19.52.124:45647 with env {KRB5CCNAME=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517687384214-19665-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517687384214-19665-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517687384214-19665-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 06:55:24.560163 26802 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.19.52.66@KRBTEST.COM'} at 127.19.52.66:54215
I20260812 06:55:24.665755 28095 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:24.659585 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.124:45647 (local address 127.0.0.1:36318)
0812 06:55:24.660005 (+ 420us) negotiation.cc:107] Waiting for socket to connect
0812 06:55:24.660020 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 06:55:24.660133 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:55:24.661070 (+ 937us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:55:24.661106 (+ 36us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:55:24.661135 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 06:55:24.661246 (+ 111us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:55:24.661252 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:55:24.662633 (+ 1381us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:55:24.662636 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:55:24.664028 (+ 1392us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:55:24.664042 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:55:24.665182 (+ 1140us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:55:24.665186 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:55:24.665330 (+ 144us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:55:24.665366 (+ 36us) client_negotiation.cc:769] Sending connection context
0812 06:55:24.665472 (+ 106us) client_negotiation.cc:240] Negotiation successful
0812 06:55:24.665596 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":326,"thread_start_us":134,"threads_started":1}
I20260812 06:55:24.665740 27927 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:24.659810 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36318 (local address 127.19.52.124:45647)
0812 06:55:24.660080 (+ 270us) server_negotiation.cc:206] Beginning negotiation
0812 06:55:24.660087 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 06:55:24.660117 (+ 30us) server_negotiation.cc:407] Connection header received
0812 06:55:24.660395 (+ 278us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:55:24.660401 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:55:24.660700 (+ 299us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:55:24.661055 (+ 355us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 06:55:24.661388 (+ 333us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:24.662456 (+ 1068us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:24.664185 (+ 1729us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:24.664989 (+ 804us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:24.665127 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:55:24.665346 (+ 219us) server_negotiation.cc:1035] Waiting for connection context
0812 06:55:24.665498 (+ 152us) server_negotiation.cc:299] Negotiation successful
0812 06:55:24.665588 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":108}
I20260812 06:55:24.666688 27723 raft_consensus.cc:1203] T 00000000000000000000000000000000 P 950b5d35439b4e23bd6d86c6664ddc43 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.6->[1.7-1.7] Dedup: 1.7->[]
I20260812 06:55:24.667181 27756 sys_catalog.cc:455] T 00000000000000000000000000000000 P 950b5d35439b4e23bd6d86c6664ddc43 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 53e6ebc3b8ee4672983fb82b50148e26. Latest consensus state: current_term: 1 leader_uuid: "53e6ebc3b8ee4672983fb82b50148e26" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "53e6ebc3b8ee4672983fb82b50148e26" member_type: VOTER last_known_addr { host: "127.19.52.126" port: 34933 } } peers { permanent_uuid: "2b5a510eb7be4cfab2f7914ebabaa929" member_type: VOTER last_known_addr { host: "127.19.52.125" port: 32823 } attrs { promote: false } } peers { permanent_uuid: "950b5d35439b4e23bd6d86c6664ddc43" member_type: NON_VOTER last_known_addr { host: "127.19.52.124" port: 45647 } attrs { promote: true } } }
I20260812 06:55:24.667333 27756 sys_catalog.cc:458] T 00000000000000000000000000000000 P 950b5d35439b4e23bd6d86c6664ddc43 [sys.catalog]: This master's current role is: LEARNER
W20260812 06:55:24.701886 28093 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:55:24.702277 28093 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:55:24.702366 28093 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 06:55:24.707302 28093 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 06:55:24.707419 28093 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 06:55:24.707527 28093 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.52.67
I20260812 06:55:24.713677 28093 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.52.84:43707
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517687384214-19665-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517687384214-19665-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517687384214-19665-0/krb5kdc/oryx_127.19.52.67.keytab
--principal=oryx/127.19.52.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.52.67:45705
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517687384214-19665-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.19.52.67
--webserver_port=38493
--tserver_master_addrs=127.19.52.126:34933,127.19.52.125:32823,127.19.52.124:45647
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28093
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.52.67
--log_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517687384214-19665-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 06:15:54 UTC on ccycloud.quasar-rnfzce.root.comops.site
build id 4581989
I20260812 06:55:24.714936 28093 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:55:24.716034 28093 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 06:55:24.726457 28101 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:55:24.726482 28104 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:55:24.726536 28093 server_base.cc:1034] running on GCE node
W20260812 06:55:24.726773 28100 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:55:24.727211 28093 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:55:24.727948 28093 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:55:24.729190 28093 hybrid_clock.cc:648] HybridClock initialized: now 1786517724729169 us; error 35 us; skew 500 ppm
Aug 12 06:55:24 dist-test-slave-65mx krb5kdc[26753](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517724, etypes {rep=17 tkt=17 ses=17}, oryx/127.19.52.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 06:55:24.732852 28093 init.cc:377] Logged in from keytab as oryx/127.19.52.67@KRBTEST.COM (short username oryx)
I20260812 06:55:24.733826 28093 webserver.cc:460] Webserver started at http://127.19.52.67:38493/ using document root <none> and password file <none>
I20260812 06:55:24.734439 28093 fs_manager.cc:359] Metadata directory not provided
I20260812 06:55:24.734549 28093 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:55:24.738960 28093 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.001s
I20260812 06:55:24.741436 28111 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 06:55:24.743026 28093 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260812 06:55:24.743166 28093 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517687384214-19665-0/minicluster-data/ts-2/data,/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517687384214-19665-0/minicluster-data/ts-2/wal
uuid: "fff2f0f0327b47368c203f7f1aab31fd"
format_stamp: "Formatted at 2026-08-12 06:55:23 on dist-test-slave-65mx"
server_key: "882f8e9953ea68aefeb9ea6311bf1b6a"
server_key_iv: "3242c8c2026901056bbe3e383384a576"
server_key_version: "encryptionkey@0"
I20260812 06:55:24.743777 28093 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517687384214-19665-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517687384214-19665-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517687384214-19665-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:55:24.785490 28093 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 06:55:24.789937 28093 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 06:55:24.790185 28093 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:55:24.790872 28093 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 06:55:24.791868 28093 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 06:55:24.791953 28093 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 06:55:24.792022 28093 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 06:55:24.792065 28093 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 06:55:24.803031 28093 rpc_server.cc:307] RPC server started. Bound to: 127.19.52.67:45705
I20260812 06:55:24.803063 28242 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.52.67:45705 every 8 connection(s)
I20260812 06:55:24.804307 28093 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517687384214-19665-0/minicluster-data/ts-2/data/info.pb
I20260812 06:55:24.811157 19665 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu as pid 28093
Aug 12 06:55:24 dist-test-slave-65mx krb5kdc[26753](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517724, etypes {rep=17 tkt=17 ses=17}, oryx/127.19.52.67@KRBTEST.COM for oryx/127.19.52.124@KRBTEST.COM
Aug 12 06:55:24 dist-test-slave-65mx krb5kdc[26753](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517724, etypes {rep=17 tkt=17 ses=17}, oryx/127.19.52.67@KRBTEST.COM for oryx/127.19.52.126@KRBTEST.COM
Aug 12 06:55:24 dist-test-slave-65mx krb5kdc[26753](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517724, etypes {rep=17 tkt=17 ses=17}, oryx/127.19.52.67@KRBTEST.COM for oryx/127.19.52.125@KRBTEST.COM
I20260812 06:55:24.827708 27645 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:24.806488 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.52.67:36039 (local address 127.19.52.125:32823)
0812 06:55:24.806632 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0812 06:55:24.806637 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 06:55:24.808042 (+ 1405us) server_negotiation.cc:407] Connection header received
0812 06:55:24.809381 (+ 1339us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:55:24.809385 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:55:24.809451 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:55:24.809545 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:55:24.811452 (+ 1907us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:24.812438 (+ 986us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:24.815171 (+ 2733us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:24.815520 (+ 349us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:24.815606 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:55:24.822515 (+ 6909us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:55:24.822521 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:55:24.822524 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:55:24.822558 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:55:24.824681 (+ 2123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:55:24.825491 (+ 810us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:55:24.825496 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:55:24.825499 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:55:24.825610 (+ 111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:55:24.826131 (+ 521us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:55:24.826137 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:55:24.826142 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:55:24.826391 (+ 249us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:55:24.826486 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0812 06:55:24.827321 (+ 835us) server_negotiation.cc:299] Negotiation successful
0812 06:55:24.827493 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260812 06:55:24.829707 28249 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:24.807003 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.125:32823 (local address 127.19.52.67:36039)
0812 06:55:24.807846 (+ 843us) negotiation.cc:107] Waiting for socket to connect
0812 06:55:24.807907 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0812 06:55:24.809144 (+ 1237us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:55:24.809748 (+ 604us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:55:24.809751 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:55:24.810199 (+ 448us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:55:24.811109 (+ 910us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:55:24.811134 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:55:24.812700 (+ 1566us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:55:24.812703 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:55:24.814991 (+ 2288us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:55:24.815004 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:55:24.816366 (+ 1362us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:55:24.816369 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:55:24.816606 (+ 237us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:55:24.817376 (+ 770us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 06:55:24.817389 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:55:24.822321 (+ 4932us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 06:55:24.824848 (+ 2527us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:55:24.824857 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:55:24.824882 (+ 25us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:55:24.825351 (+ 469us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:55:24.825753 (+ 402us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:55:24.825757 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:55:24.825760 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:55:24.826002 (+ 242us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:55:24.826553 (+ 551us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 06:55:24.826564 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 06:55:24.827006 (+ 442us) client_negotiation.cc:769] Sending connection context
0812 06:55:24.828011 (+ 1005us) client_negotiation.cc:240] Negotiation successful
0812 06:55:24.828443 (+ 432us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":625,"mutex_wait_us":878,"thread_start_us":696,"threads_started":1}
I20260812 06:55:24.830034 27927 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:24.807250 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.52.67:47815 (local address 127.19.52.124:45647)
0812 06:55:24.807386 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0812 06:55:24.807391 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 06:55:24.808758 (+ 1367us) server_negotiation.cc:407] Connection header received
0812 06:55:24.809395 (+ 637us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:55:24.809399 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:55:24.809468 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:55:24.809522 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:55:24.811303 (+ 1781us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:24.812297 (+ 994us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:24.815271 (+ 2974us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:24.815615 (+ 344us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:24.815694 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:55:24.821619 (+ 5925us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:55:24.821625 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:55:24.821629 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:55:24.821670 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:55:24.825185 (+ 3515us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:55:24.827678 (+ 2493us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:55:24.827682 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:55:24.827684 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:55:24.827811 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:55:24.828817 (+ 1006us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:55:24.828821 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:55:24.828823 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:55:24.829048 (+ 225us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:55:24.829135 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0812 06:55:24.829742 (+ 607us) server_negotiation.cc:299] Negotiation successful
0812 06:55:24.829875 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56,"spinlock_wait_cycles":896}
I20260812 06:55:24.830821 28256 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:24.814820 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44494 (local address 127.19.52.126:34933)
0812 06:55:24.816733 (+ 1913us) server_negotiation.cc:206] Beginning negotiation
0812 06:55:24.816739 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 06:55:24.816756 (+ 17us) server_negotiation.cc:407] Connection header received
0812 06:55:24.816806 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:55:24.816811 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:55:24.816875 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:55:24.817004 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:55:24.818622 (+ 1618us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:24.819458 (+ 836us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:24.823465 (+ 4007us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:24.823848 (+ 383us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:24.823972 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:55:24.825728 (+ 1756us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:55:24.825734 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:55:24.825737 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:55:24.825769 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:55:24.828174 (+ 2405us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:55:24.828783 (+ 609us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:55:24.828788 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:55:24.828789 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:55:24.828886 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:55:24.829307 (+ 421us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:55:24.829312 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:55:24.829315 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:55:24.829635 (+ 320us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:55:24.829782 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0812 06:55:24.830463 (+ 681us) server_negotiation.cc:299] Negotiation successful
0812 06:55:24.830621 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1781,"thread_start_us":110,"threads_started":1}
I20260812 06:55:24.831599 28247 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:24.807003 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.124:45647 (local address 127.19.52.67:47815)
0812 06:55:24.808660 (+ 1657us) negotiation.cc:107] Waiting for socket to connect
0812 06:55:24.808671 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 06:55:24.809171 (+ 500us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:55:24.809740 (+ 569us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:55:24.809751 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:55:24.810349 (+ 598us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:55:24.811109 (+ 760us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:55:24.811132 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:55:24.812700 (+ 1568us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:55:24.812703 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:55:24.814980 (+ 2277us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:55:24.815004 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:55:24.815905 (+ 901us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:55:24.815909 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:55:24.816162 (+ 253us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:55:24.817175 (+ 1013us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 06:55:24.817202 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:55:24.821313 (+ 4111us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 06:55:24.826477 (+ 5164us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:55:24.826480 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:55:24.826482 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:55:24.826750 (+ 268us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:55:24.828164 (+ 1414us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:55:24.828167 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:55:24.828169 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:55:24.828256 (+ 87us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:55:24.829442 (+ 1186us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 06:55:24.829445 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 06:55:24.829531 (+ 86us) client_negotiation.cc:769] Sending connection context
0812 06:55:24.831155 (+ 1624us) client_negotiation.cc:240] Negotiation successful
0812 06:55:24.831337 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1569,"mutex_wait_us":992,"spinlock_wait_cycles":111232,"thread_start_us":130,"threads_started":1}
I20260812 06:55:24.831910 28245 heartbeater.cc:347] Connected to a master server at 127.19.52.125:32823
I20260812 06:55:24.832211 28245 heartbeater.cc:464] Registering TS with master...
I20260812 06:55:24.832233 26848 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:24.808353 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.52.67:51625 (local address 127.19.52.126:34933)
0812 06:55:24.808526 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0812 06:55:24.808532 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 06:55:24.808574 (+ 42us) server_negotiation.cc:407] Connection header received
0812 06:55:24.809396 (+ 822us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:55:24.809399 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:55:24.809456 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:55:24.809513 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:55:24.811304 (+ 1791us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:24.812296 (+ 992us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:24.815757 (+ 3461us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:24.816090 (+ 333us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:24.816195 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:55:24.823144 (+ 6949us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:55:24.823151 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:55:24.823157 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:55:24.823193 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:55:24.826772 (+ 3579us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:55:24.827215 (+ 443us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:55:24.827222 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:55:24.827224 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:55:24.827321 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:55:24.828379 (+ 1058us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:55:24.828386 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:55:24.828389 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:55:24.828647 (+ 258us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:55:24.828737 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0812 06:55:24.831874 (+ 3137us) server_negotiation.cc:299] Negotiation successful
0812 06:55:24.832008 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260812 06:55:24.832808 28250 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:24.807003 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.126:34933 (local address 127.19.52.67:51625)
0812 06:55:24.808352 (+ 1349us) negotiation.cc:107] Waiting for socket to connect
0812 06:55:24.808363 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 06:55:24.809171 (+ 808us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:55:24.809740 (+ 569us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:55:24.809751 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:55:24.810349 (+ 598us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:55:24.811109 (+ 760us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:55:24.811133 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:55:24.812700 (+ 1567us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:55:24.812703 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:55:24.815134 (+ 2431us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:55:24.815143 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:55:24.816899 (+ 1756us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:55:24.816918 (+ 19us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:55:24.817049 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:55:24.818024 (+ 975us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 06:55:24.818045 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:55:24.821993 (+ 3948us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 06:55:24.826858 (+ 4865us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:55:24.826861 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:55:24.826863 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:55:24.827087 (+ 224us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:55:24.827464 (+ 377us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:55:24.827467 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:55:24.827469 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:55:24.827555 (+ 86us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:55:24.829196 (+ 1641us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 06:55:24.829199 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 06:55:24.829307 (+ 108us) client_negotiation.cc:769] Sending connection context
0812 06:55:24.832422 (+ 3115us) client_negotiation.cc:240] Negotiation successful
0812 06:55:24.832630 (+ 208us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1269,"mutex_wait_us":570,"spinlock_wait_cycles":16512,"thread_start_us":171,"threads_started":1}
I20260812 06:55:24.832839 28245 heartbeater.cc:511] Master 127.19.52.125:32823 requested a full tablet report, sending...
I20260812 06:55:24.833215 28243 heartbeater.cc:347] Connected to a master server at 127.19.52.124:45647
I20260812 06:55:24.833299 28243 heartbeater.cc:464] Registering TS with master...
I20260812 06:55:24.833429 28244 heartbeater.cc:347] Connected to a master server at 127.19.52.126:34933
I20260812 06:55:24.833482 28243 heartbeater.cc:511] Master 127.19.52.124:45647 requested a full tablet report, sending...
I20260812 06:55:24.833540 28244 heartbeater.cc:464] Registering TS with master...
I20260812 06:55:24.833732 28244 heartbeater.cc:511] Master 127.19.52.126:34933 requested a full tablet report, sending...
I20260812 06:55:24.834378 26802 ts_manager.cc:194] Re-registered known tserver with Master: fff2f0f0327b47368c203f7f1aab31fd (127.19.52.67:45705)
I20260812 06:55:24.834409 27503 ts_manager.cc:194] Registered new tserver with Master: fff2f0f0327b47368c203f7f1aab31fd (127.19.52.67:45705)
I20260812 06:55:24.834532 27712 ts_manager.cc:194] Registered new tserver with Master: fff2f0f0327b47368c203f7f1aab31fd (127.19.52.67:45705)
I20260812 06:55:24.834697 28264 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:24.815820 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36322 (local address 127.19.52.124:45647)
0812 06:55:24.818534 (+ 2714us) server_negotiation.cc:206] Beginning negotiation
0812 06:55:24.818552 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0812 06:55:24.819898 (+ 1346us) server_negotiation.cc:407] Connection header received
0812 06:55:24.819948 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:55:24.819951 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:55:24.820002 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:55:24.820089 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:55:24.821109 (+ 1020us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:24.821882 (+ 773us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:24.823224 (+ 1342us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:24.823572 (+ 348us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:24.823665 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:55:24.827652 (+ 3987us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:55:24.827659 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:55:24.827664 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:55:24.827723 (+ 59us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:55:24.831279 (+ 3556us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:55:24.831918 (+ 639us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:55:24.831922 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:55:24.831923 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:55:24.832055 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:55:24.832497 (+ 442us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:55:24.832500 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:55:24.832502 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:55:24.832768 (+ 266us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:55:24.833621 (+ 853us) server_negotiation.cc:1035] Waiting for connection context
0812 06:55:24.833900 (+ 279us) server_negotiation.cc:299] Negotiation successful
0812 06:55:24.834065 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2616,"spinlock_wait_cycles":13568,"thread_start_us":90,"threads_started":1}
I20260812 06:55:24.835098 26802 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.19.52.67@KRBTEST.COM'} at 127.19.52.67:51625
I20260812 06:55:24.837345 28259 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:24.815167 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55278 (local address 127.19.52.125:32823)
0812 06:55:24.816359 (+ 1192us) server_negotiation.cc:206] Beginning negotiation
0812 06:55:24.816364 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 06:55:24.818085 (+ 1721us) server_negotiation.cc:407] Connection header received
0812 06:55:24.818151 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:55:24.818154 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:55:24.818208 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:55:24.818299 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:55:24.819404 (+ 1105us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:24.820249 (+ 845us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:24.821501 (+ 1252us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:24.821866 (+ 365us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:24.822564 (+ 698us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:55:24.825470 (+ 2906us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:55:24.825474 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:55:24.825477 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:55:24.825510 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:55:24.829213 (+ 3703us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:55:24.830722 (+ 1509us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:55:24.830726 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:55:24.830728 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:55:24.830873 (+ 145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:55:24.835945 (+ 5072us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:55:24.835956 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:55:24.835958 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:55:24.836198 (+ 240us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:55:24.836339 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0812 06:55:24.837044 (+ 705us) server_negotiation.cc:299] Negotiation successful
0812 06:55:24.837195 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1107,"spinlock_wait_cycles":5888,"thread_start_us":82,"threads_started":1}
I20260812 06:55:24.836436 26802 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44494:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 06:55:24.840302 26802 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 06:55:24.843394 27453 sys_catalog.cc:455] T 00000000000000000000000000000000 P 53e6ebc3b8ee4672983fb82b50148e26 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "53e6ebc3b8ee4672983fb82b50148e26" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "53e6ebc3b8ee4672983fb82b50148e26" member_type: VOTER last_known_addr { host: "127.19.52.126" port: 34933 } } peers { permanent_uuid: "2b5a510eb7be4cfab2f7914ebabaa929" member_type: VOTER last_known_addr { host: "127.19.52.125" port: 32823 } attrs { promote: false } } peers { permanent_uuid: "950b5d35439b4e23bd6d86c6664ddc43" member_type: NON_VOTER last_known_addr { host: "127.19.52.124" port: 45647 } attrs { promote: true } } }
I20260812 06:55:24.843592 27453 sys_catalog.cc:458] T 00000000000000000000000000000000 P 53e6ebc3b8ee4672983fb82b50148e26 [sys.catalog]: This master's current role is: LEADER
I20260812 06:55:24.847366 27453 raft_consensus.cc:1050] T 00000000000000000000000000000000 P 53e6ebc3b8ee4672983fb82b50148e26: attempting to promote NON_VOTER 950b5d35439b4e23bd6d86c6664ddc43 to VOTER
I20260812 06:55:24.847797 27453 consensus_queue.cc:237] T 00000000000000000000000000000000 P 53e6ebc3b8ee4672983fb82b50148e26 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "53e6ebc3b8ee4672983fb82b50148e26" member_type: VOTER last_known_addr { host: "127.19.52.126" port: 34933 } } peers { permanent_uuid: "2b5a510eb7be4cfab2f7914ebabaa929" member_type: VOTER last_known_addr { host: "127.19.52.125" port: 32823 } attrs { promote: false } } peers { permanent_uuid: "950b5d35439b4e23bd6d86c6664ddc43" member_type: VOTER last_known_addr { host: "127.19.52.124" port: 45647 } attrs { promote: false } }
I20260812 06:55:24.848950 27723 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 950b5d35439b4e23bd6d86c6664ddc43 [term 1 LEARNER]: Refusing update from remote peer 53e6ebc3b8ee4672983fb82b50148e26: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20260812 06:55:24.849330 28270 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 53e6ebc3b8ee4672983fb82b50148e26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "950b5d35439b4e23bd6d86c6664ddc43" member_type: VOTER last_known_addr { host: "127.19.52.124" port: 45647 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260812 06:55:24.849377 27514 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 2b5a510eb7be4cfab2f7914ebabaa929 [term 1 FOLLOWER]: Refusing update from remote peer 53e6ebc3b8ee4672983fb82b50148e26: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20260812 06:55:24.849858 28270 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 53e6ebc3b8ee4672983fb82b50148e26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2b5a510eb7be4cfab2f7914ebabaa929" member_type: VOTER last_known_addr { host: "127.19.52.125" port: 32823 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260812 06:55:24.851611 28269 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 53e6ebc3b8ee4672983fb82b50148e26 [term 1 LEADER]: Committing config change with OpId 1.9: config changed from index 7 to 9, 950b5d35439b4e23bd6d86c6664ddc43 (127.19.52.124) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "53e6ebc3b8ee4672983fb82b50148e26" member_type: VOTER last_known_addr { host: "127.19.52.126" port: 34933 } } peers { permanent_uuid: "2b5a510eb7be4cfab2f7914ebabaa929" member_type: VOTER last_known_addr { host: "127.19.52.125" port: 32823 } attrs { promote: false } } peers { permanent_uuid: "950b5d35439b4e23bd6d86c6664ddc43" member_type: VOTER last_known_addr { host: "127.19.52.124" port: 45647 } attrs { promote: false } } }
I20260812 06:55:24.852878 27723 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 950b5d35439b4e23bd6d86c6664ddc43 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 7 to 9, 950b5d35439b4e23bd6d86c6664ddc43 (127.19.52.124) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "53e6ebc3b8ee4672983fb82b50148e26" member_type: VOTER last_known_addr { host: "127.19.52.126" port: 34933 } } peers { permanent_uuid: "2b5a510eb7be4cfab2f7914ebabaa929" member_type: VOTER last_known_addr { host: "127.19.52.125" port: 32823 } attrs { promote: false } } peers { permanent_uuid: "950b5d35439b4e23bd6d86c6664ddc43" member_type: VOTER last_known_addr { host: "127.19.52.124" port: 45647 } attrs { promote: false } } }
I20260812 06:55:24.853852 28269 sys_catalog.cc:455] T 00000000000000000000000000000000 P 53e6ebc3b8ee4672983fb82b50148e26 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "53e6ebc3b8ee4672983fb82b50148e26" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "53e6ebc3b8ee4672983fb82b50148e26" member_type: VOTER last_known_addr { host: "127.19.52.126" port: 34933 } } peers { permanent_uuid: "2b5a510eb7be4cfab2f7914ebabaa929" member_type: VOTER last_known_addr { host: "127.19.52.125" port: 32823 } attrs { promote: false } } peers { permanent_uuid: "950b5d35439b4e23bd6d86c6664ddc43" member_type: VOTER last_known_addr { host: "127.19.52.124" port: 45647 } attrs { promote: false } } }
I20260812 06:55:24.853960 28269 sys_catalog.cc:458] T 00000000000000000000000000000000 P 53e6ebc3b8ee4672983fb82b50148e26 [sys.catalog]: This master's current role is: LEADER
I20260812 06:55:24.853891 27514 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 2b5a510eb7be4cfab2f7914ebabaa929 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 7 to 9, 950b5d35439b4e23bd6d86c6664ddc43 (127.19.52.124) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "53e6ebc3b8ee4672983fb82b50148e26" member_type: VOTER last_known_addr { host: "127.19.52.126" port: 34933 } } peers { permanent_uuid: "2b5a510eb7be4cfab2f7914ebabaa929" member_type: VOTER last_known_addr { host: "127.19.52.125" port: 32823 } attrs { promote: false } } peers { permanent_uuid: "950b5d35439b4e23bd6d86c6664ddc43" member_type: VOTER last_known_addr { host: "127.19.52.124" port: 45647 } attrs { promote: false } } }
I20260812 06:55:24.854151 28269 sys_catalog.cc:455] T 00000000000000000000000000000000 P 53e6ebc3b8ee4672983fb82b50148e26 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "53e6ebc3b8ee4672983fb82b50148e26" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "53e6ebc3b8ee4672983fb82b50148e26" member_type: VOTER last_known_addr { host: "127.19.52.126" port: 34933 } } peers { permanent_uuid: "2b5a510eb7be4cfab2f7914ebabaa929" member_type: VOTER last_known_addr { host: "127.19.52.125" port: 32823 } attrs { promote: false } } peers { permanent_uuid: "950b5d35439b4e23bd6d86c6664ddc43" member_type: VOTER last_known_addr { host: "127.19.52.124" port: 45647 } attrs { promote: false } } }
I20260812 06:55:24.854223 28269 sys_catalog.cc:458] T 00000000000000000000000000000000 P 53e6ebc3b8ee4672983fb82b50148e26 [sys.catalog]: This master's current role is: LEADER
I20260812 06:55:24.854338 28269 sys_catalog.cc:455] T 00000000000000000000000000000000 P 53e6ebc3b8ee4672983fb82b50148e26 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "53e6ebc3b8ee4672983fb82b50148e26" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "53e6ebc3b8ee4672983fb82b50148e26" member_type: VOTER last_known_addr { host: "127.19.52.126" port: 34933 } } peers { permanent_uuid: "2b5a510eb7be4cfab2f7914ebabaa929" member_type: VOTER last_known_addr { host: "127.19.52.125" port: 32823 } attrs { promote: false } } peers { permanent_uuid: "950b5d35439b4e23bd6d86c6664ddc43" member_type: VOTER last_known_addr { host: "127.19.52.124" port: 45647 } attrs { promote: false } } }
I20260812 06:55:24.854429 28269 sys_catalog.cc:458] T 00000000000000000000000000000000 P 53e6ebc3b8ee4672983fb82b50148e26 [sys.catalog]: This master's current role is: LEADER
I20260812 06:55:24.855602 27756 sys_catalog.cc:455] T 00000000000000000000000000000000 P 950b5d35439b4e23bd6d86c6664ddc43 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "53e6ebc3b8ee4672983fb82b50148e26" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "53e6ebc3b8ee4672983fb82b50148e26" member_type: VOTER last_known_addr { host: "127.19.52.126" port: 34933 } } peers { permanent_uuid: "2b5a510eb7be4cfab2f7914ebabaa929" member_type: VOTER last_known_addr { host: "127.19.52.125" port: 32823 } attrs { promote: false } } peers { permanent_uuid: "950b5d35439b4e23bd6d86c6664ddc43" member_type: VOTER last_known_addr { host: "127.19.52.124" port: 45647 } attrs { promote: false } } }
I20260812 06:55:24.855723 27756 sys_catalog.cc:458] T 00000000000000000000000000000000 P 950b5d35439b4e23bd6d86c6664ddc43 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 06:55:24.856375 28279 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2b5a510eb7be4cfab2f7914ebabaa929 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "53e6ebc3b8ee4672983fb82b50148e26" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "53e6ebc3b8ee4672983fb82b50148e26" member_type: VOTER last_known_addr { host: "127.19.52.126" port: 34933 } } peers { permanent_uuid: "2b5a510eb7be4cfab2f7914ebabaa929" member_type: VOTER last_known_addr { host: "127.19.52.125" port: 32823 } attrs { promote: false } } peers { permanent_uuid: "950b5d35439b4e23bd6d86c6664ddc43" member_type: VOTER last_known_addr { host: "127.19.52.124" port: 45647 } attrs { promote: false } } }
I20260812 06:55:24.857983 28279 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2b5a510eb7be4cfab2f7914ebabaa929 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 06:55:24.865553 28095 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:24.859308 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.66:34045 (local address 127.0.0.1:43876)
0812 06:55:24.859487 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0812 06:55:24.859505 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0812 06:55:24.859594 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:55:24.861060 (+ 1466us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:55:24.861064 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:55:24.861082 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 06:55:24.861214 (+ 132us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:55:24.861221 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:55:24.862794 (+ 1573us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:55:24.862798 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:55:24.863936 (+ 1138us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:55:24.863947 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:55:24.865066 (+ 1119us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:55:24.865069 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:55:24.865152 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:55:24.865178 (+ 26us) client_negotiation.cc:769] Sending connection context
0812 06:55:24.865250 (+ 72us) client_negotiation.cc:240] Negotiation successful
0812 06:55:24.865339 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":94,"mutex_wait_us":1}
I20260812 06:55:24.865963 28280 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:24.859693 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.65:41417 (local address 127.0.0.1:36754)
0812 06:55:24.860091 (+ 398us) negotiation.cc:107] Waiting for socket to connect
0812 06:55:24.860109 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0812 06:55:24.860205 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:55:24.861614 (+ 1409us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:55:24.861618 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:55:24.861642 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 06:55:24.861746 (+ 104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:55:24.861752 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:55:24.863221 (+ 1469us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:55:24.863225 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:55:24.864353 (+ 1128us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:55:24.864362 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:55:24.865498 (+ 1136us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:55:24.865502 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:55:24.865617 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:55:24.865643 (+ 26us) client_negotiation.cc:769] Sending connection context
0812 06:55:24.865702 (+ 59us) client_negotiation.cc:240] Negotiation successful
0812 06:55:24.865794 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":317,"thread_start_us":160,"threads_started":1}
I20260812 06:55:24.866439 28281 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:24.859963 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.67:45705 (local address 127.0.0.1:33194)
0812 06:55:24.861089 (+ 1126us) negotiation.cc:107] Waiting for socket to connect
0812 06:55:24.861102 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 06:55:24.861234 (+ 132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:55:24.862099 (+ 865us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:55:24.862103 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:55:24.862122 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 06:55:24.862215 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:55:24.862221 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:55:24.863705 (+ 1484us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:55:24.863710 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:55:24.864817 (+ 1107us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:55:24.864828 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:55:24.866038 (+ 1210us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:55:24.866042 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:55:24.866130 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:55:24.866152 (+ 22us) client_negotiation.cc:769] Sending connection context
0812 06:55:24.866231 (+ 79us) client_negotiation.cc:240] Negotiation successful
0812 06:55:24.866310 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1070,"mutex_wait_us":36,"thread_start_us":102,"threads_started":1}
I20260812 06:55:24.866549 28287 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:24.860005 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43876 (local address 127.19.52.66:34045)
0812 06:55:24.860395 (+ 390us) server_negotiation.cc:206] Beginning negotiation
0812 06:55:24.860411 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0812 06:55:24.860440 (+ 29us) server_negotiation.cc:407] Connection header received
0812 06:55:24.860574 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:55:24.860585 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:55:24.860868 (+ 283us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:55:24.861092 (+ 224us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 06:55:24.861331 (+ 239us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:24.862645 (+ 1314us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:24.864090 (+ 1445us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:24.864941 (+ 851us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:24.865056 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:55:24.866211 (+ 1155us) server_negotiation.cc:1035] Waiting for connection context
0812 06:55:24.866271 (+ 60us) server_negotiation.cc:299] Negotiation successful
0812 06:55:24.866385 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":262,"thread_start_us":145,"threads_started":1}
I20260812 06:55:24.866890 28289 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:24.860683 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36754 (local address 127.19.52.65:41417)
0812 06:55:24.861027 (+ 344us) server_negotiation.cc:206] Beginning negotiation
0812 06:55:24.861033 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 06:55:24.861057 (+ 24us) server_negotiation.cc:407] Connection header received
0812 06:55:24.861132 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:55:24.861137 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:55:24.861326 (+ 189us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:55:24.861572 (+ 246us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 06:55:24.861890 (+ 318us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:24.863086 (+ 1196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:24.864491 (+ 1405us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:24.865358 (+ 867us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:24.866433 (+ 1075us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:55:24.866602 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0812 06:55:24.866663 (+ 61us) server_negotiation.cc:299] Negotiation successful
0812 06:55:24.866765 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":212,"thread_start_us":117,"threads_started":1}
I20260812 06:55:24.866942 28288 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:24.860396 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33194 (local address 127.19.52.67:45705)
0812 06:55:24.861629 (+ 1233us) server_negotiation.cc:206] Beginning negotiation
0812 06:55:24.861644 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0812 06:55:24.861666 (+ 22us) server_negotiation.cc:407] Connection header received
0812 06:55:24.861738 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:55:24.861742 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:55:24.861920 (+ 178us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:55:24.862421 (+ 501us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 06:55:24.862563 (+ 142us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:24.863585 (+ 1022us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:24.864970 (+ 1385us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:24.865711 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:24.865815 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:55:24.865971 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0812 06:55:24.866726 (+ 755us) server_negotiation.cc:299] Negotiation successful
0812 06:55:24.866827 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1028,"thread_start_us":188,"threads_started":1}
I20260812 06:55:24.869577 28167 tablet_service.cc:1467] Processing CreateTablet for tablet b4ccea11ed5e4cc4ad6f007cb4ca467b (DEFAULT_TABLE table=test-table [id=86d310dc8a2b47d4877ccdd7d6582b24]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 06:55:24.869609 28011 tablet_service.cc:1467] Processing CreateTablet for tablet b4ccea11ed5e4cc4ad6f007cb4ca467b (DEFAULT_TABLE table=test-table [id=86d310dc8a2b47d4877ccdd7d6582b24]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 06:55:24.870579 28167 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b4ccea11ed5e4cc4ad6f007cb4ca467b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 06:55:24.870682 28011 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b4ccea11ed5e4cc4ad6f007cb4ca467b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 06:55:24.870597 27843 tablet_service.cc:1467] Processing CreateTablet for tablet b4ccea11ed5e4cc4ad6f007cb4ca467b (DEFAULT_TABLE table=test-table [id=86d310dc8a2b47d4877ccdd7d6582b24]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 06:55:24.871630 27843 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b4ccea11ed5e4cc4ad6f007cb4ca467b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 06:55:24.880443 28291 tablet_bootstrap.cc:492] T b4ccea11ed5e4cc4ad6f007cb4ca467b P 2e97338a875f4bada5bf4e6301558f78: Bootstrap starting.
I20260812 06:55:24.880431 28290 tablet_bootstrap.cc:492] T b4ccea11ed5e4cc4ad6f007cb4ca467b P fff2f0f0327b47368c203f7f1aab31fd: Bootstrap starting.
I20260812 06:55:24.881359 28293 tablet_bootstrap.cc:492] T b4ccea11ed5e4cc4ad6f007cb4ca467b P 49e259bbf4094665b4e899332664889c: Bootstrap starting.
I20260812 06:55:24.883299 28290 tablet_bootstrap.cc:654] T b4ccea11ed5e4cc4ad6f007cb4ca467b P fff2f0f0327b47368c203f7f1aab31fd: Neither blocks nor log segments found. Creating new log.
I20260812 06:55:24.883615 28293 tablet_bootstrap.cc:654] T b4ccea11ed5e4cc4ad6f007cb4ca467b P 49e259bbf4094665b4e899332664889c: Neither blocks nor log segments found. Creating new log.
I20260812 06:55:24.884382 28290 log.cc:824] T b4ccea11ed5e4cc4ad6f007cb4ca467b P fff2f0f0327b47368c203f7f1aab31fd: Log is configured to *not* fsync() on all Append() calls
I20260812 06:55:24.884425 28293 log.cc:824] T b4ccea11ed5e4cc4ad6f007cb4ca467b P 49e259bbf4094665b4e899332664889c: Log is configured to *not* fsync() on all Append() calls
I20260812 06:55:24.886968 28290 tablet_bootstrap.cc:492] T b4ccea11ed5e4cc4ad6f007cb4ca467b P fff2f0f0327b47368c203f7f1aab31fd: No bootstrap required, opened a new log
I20260812 06:55:24.887102 28293 tablet_bootstrap.cc:492] T b4ccea11ed5e4cc4ad6f007cb4ca467b P 49e259bbf4094665b4e899332664889c: No bootstrap required, opened a new log
I20260812 06:55:24.887238 28290 ts_tablet_manager.cc:1397] T b4ccea11ed5e4cc4ad6f007cb4ca467b P fff2f0f0327b47368c203f7f1aab31fd: Time spent bootstrapping tablet: real 0.007s user 0.002s sys 0.002s
I20260812 06:55:24.887319 28293 ts_tablet_manager.cc:1397] T b4ccea11ed5e4cc4ad6f007cb4ca467b P 49e259bbf4094665b4e899332664889c: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260812 06:55:24.887401 28291 tablet_bootstrap.cc:654] T b4ccea11ed5e4cc4ad6f007cb4ca467b P 2e97338a875f4bada5bf4e6301558f78: Neither blocks nor log segments found. Creating new log.
I20260812 06:55:24.888362 28291 log.cc:824] T b4ccea11ed5e4cc4ad6f007cb4ca467b P 2e97338a875f4bada5bf4e6301558f78: Log is configured to *not* fsync() on all Append() calls
I20260812 06:55:24.891059 28291 tablet_bootstrap.cc:492] T b4ccea11ed5e4cc4ad6f007cb4ca467b P 2e97338a875f4bada5bf4e6301558f78: No bootstrap required, opened a new log
I20260812 06:55:24.891305 28291 ts_tablet_manager.cc:1397] T b4ccea11ed5e4cc4ad6f007cb4ca467b P 2e97338a875f4bada5bf4e6301558f78: Time spent bootstrapping tablet: real 0.011s user 0.002s sys 0.002s
I20260812 06:55:24.891155 28290 raft_consensus.cc:348] T b4ccea11ed5e4cc4ad6f007cb4ca467b P fff2f0f0327b47368c203f7f1aab31fd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e97338a875f4bada5bf4e6301558f78" member_type: VOTER last_known_addr { host: "127.19.52.66" port: 34045 } } peers { permanent_uuid: "49e259bbf4094665b4e899332664889c" member_type: VOTER last_known_addr { host: "127.19.52.65" port: 41417 } } peers { permanent_uuid: "fff2f0f0327b47368c203f7f1aab31fd" member_type: VOTER last_known_addr { host: "127.19.52.67" port: 45705 } }
I20260812 06:55:24.891472 28290 raft_consensus.cc:374] T b4ccea11ed5e4cc4ad6f007cb4ca467b P fff2f0f0327b47368c203f7f1aab31fd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 06:55:24.891535 28290 raft_consensus.cc:729] T b4ccea11ed5e4cc4ad6f007cb4ca467b P fff2f0f0327b47368c203f7f1aab31fd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fff2f0f0327b47368c203f7f1aab31fd, State: Initialized, Role: FOLLOWER
I20260812 06:55:24.891701 28293 raft_consensus.cc:348] T b4ccea11ed5e4cc4ad6f007cb4ca467b P 49e259bbf4094665b4e899332664889c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e97338a875f4bada5bf4e6301558f78" member_type: VOTER last_known_addr { host: "127.19.52.66" port: 34045 } } peers { permanent_uuid: "49e259bbf4094665b4e899332664889c" member_type: VOTER last_known_addr { host: "127.19.52.65" port: 41417 } } peers { permanent_uuid: "fff2f0f0327b47368c203f7f1aab31fd" member_type: VOTER last_known_addr { host: "127.19.52.67" port: 45705 } }
I20260812 06:55:24.891988 28293 raft_consensus.cc:374] T b4ccea11ed5e4cc4ad6f007cb4ca467b P 49e259bbf4094665b4e899332664889c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 06:55:24.892050 28293 raft_consensus.cc:729] T b4ccea11ed5e4cc4ad6f007cb4ca467b P 49e259bbf4094665b4e899332664889c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 49e259bbf4094665b4e899332664889c, State: Initialized, Role: FOLLOWER
I20260812 06:55:24.892095 28290 consensus_queue.cc:260] T b4ccea11ed5e4cc4ad6f007cb4ca467b P fff2f0f0327b47368c203f7f1aab31fd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e97338a875f4bada5bf4e6301558f78" member_type: VOTER last_known_addr { host: "127.19.52.66" port: 34045 } } peers { permanent_uuid: "49e259bbf4094665b4e899332664889c" member_type: VOTER last_known_addr { host: "127.19.52.65" port: 41417 } } peers { permanent_uuid: "fff2f0f0327b47368c203f7f1aab31fd" member_type: VOTER last_known_addr { host: "127.19.52.67" port: 45705 } }
I20260812 06:55:24.892688 28293 consensus_queue.cc:260] T b4ccea11ed5e4cc4ad6f007cb4ca467b P 49e259bbf4094665b4e899332664889c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e97338a875f4bada5bf4e6301558f78" member_type: VOTER last_known_addr { host: "127.19.52.66" port: 34045 } } peers { permanent_uuid: "49e259bbf4094665b4e899332664889c" member_type: VOTER last_known_addr { host: "127.19.52.65" port: 41417 } } peers { permanent_uuid: "fff2f0f0327b47368c203f7f1aab31fd" member_type: VOTER last_known_addr { host: "127.19.52.67" port: 45705 } }
I20260812 06:55:24.893253 28290 ts_tablet_manager.cc:1428] T b4ccea11ed5e4cc4ad6f007cb4ca467b P fff2f0f0327b47368c203f7f1aab31fd: Time spent starting tablet: real 0.006s user 0.006s sys 0.001s
I20260812 06:55:24.893674 28244 heartbeater.cc:503] Master 127.19.52.126:34933 was elected leader, sending a full tablet report...
I20260812 06:55:24.896714 28293 ts_tablet_manager.cc:1428] T b4ccea11ed5e4cc4ad6f007cb4ca467b P 49e259bbf4094665b4e899332664889c: Time spent starting tablet: real 0.009s user 0.008s sys 0.000s
I20260812 06:55:24.897046 27924 heartbeater.cc:503] Master 127.19.52.126:34933 was elected leader, sending a full tablet report...
I20260812 06:55:24.897382 28291 raft_consensus.cc:348] T b4ccea11ed5e4cc4ad6f007cb4ca467b P 2e97338a875f4bada5bf4e6301558f78 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e97338a875f4bada5bf4e6301558f78" member_type: VOTER last_known_addr { host: "127.19.52.66" port: 34045 } } peers { permanent_uuid: "49e259bbf4094665b4e899332664889c" member_type: VOTER last_known_addr { host: "127.19.52.65" port: 41417 } } peers { permanent_uuid: "fff2f0f0327b47368c203f7f1aab31fd" member_type: VOTER last_known_addr { host: "127.19.52.67" port: 45705 } }
I20260812 06:55:24.897682 28291 raft_consensus.cc:374] T b4ccea11ed5e4cc4ad6f007cb4ca467b P 2e97338a875f4bada5bf4e6301558f78 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 06:55:24.897751 28291 raft_consensus.cc:729] T b4ccea11ed5e4cc4ad6f007cb4ca467b P 2e97338a875f4bada5bf4e6301558f78 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2e97338a875f4bada5bf4e6301558f78, State: Initialized, Role: FOLLOWER
I20260812 06:55:24.898278 28291 consensus_queue.cc:260] T b4ccea11ed5e4cc4ad6f007cb4ca467b P 2e97338a875f4bada5bf4e6301558f78 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e97338a875f4bada5bf4e6301558f78" member_type: VOTER last_known_addr { host: "127.19.52.66" port: 34045 } } peers { permanent_uuid: "49e259bbf4094665b4e899332664889c" member_type: VOTER last_known_addr { host: "127.19.52.65" port: 41417 } } peers { permanent_uuid: "fff2f0f0327b47368c203f7f1aab31fd" member_type: VOTER last_known_addr { host: "127.19.52.67" port: 45705 } }
I20260812 06:55:24.899448 28291 ts_tablet_manager.cc:1428] T b4ccea11ed5e4cc4ad6f007cb4ca467b P 2e97338a875f4bada5bf4e6301558f78: Time spent starting tablet: real 0.008s user 0.004s sys 0.004s
I20260812 06:55:24.899845 28086 heartbeater.cc:503] Master 127.19.52.126:34933 was elected leader, sending a full tablet report...
I20260812 06:55:24.924678 28298 raft_consensus.cc:482] T b4ccea11ed5e4cc4ad6f007cb4ca467b P 49e259bbf4094665b4e899332664889c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 06:55:24.924924 28298 raft_consensus.cc:504] T b4ccea11ed5e4cc4ad6f007cb4ca467b P 49e259bbf4094665b4e899332664889c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e97338a875f4bada5bf4e6301558f78" member_type: VOTER last_known_addr { host: "127.19.52.66" port: 34045 } } peers { permanent_uuid: "49e259bbf4094665b4e899332664889c" member_type: VOTER last_known_addr { host: "127.19.52.65" port: 41417 } } peers { permanent_uuid: "fff2f0f0327b47368c203f7f1aab31fd" member_type: VOTER last_known_addr { host: "127.19.52.67" port: 45705 } }
I20260812 06:55:24.926525 28298 leader_election.cc:288] T b4ccea11ed5e4cc4ad6f007cb4ca467b P 49e259bbf4094665b4e899332664889c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2e97338a875f4bada5bf4e6301558f78 (127.19.52.66:34045), fff2f0f0327b47368c203f7f1aab31fd (127.19.52.67:45705)
I20260812 06:55:24.932907 28288 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:24.927006 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.52.65:48881 (local address 127.19.52.67:45705)
0812 06:55:24.927168 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0812 06:55:24.927172 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 06:55:24.927313 (+ 141us) server_negotiation.cc:407] Connection header received
0812 06:55:24.927420 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:55:24.927423 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:55:24.927485 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:55:24.927588 (+ 103us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 06:55:24.928173 (+ 585us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:24.929101 (+ 928us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:24.931365 (+ 2264us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:24.932097 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:24.932255 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:55:24.932314 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0812 06:55:24.932680 (+ 366us) server_negotiation.cc:299] Negotiation successful
0812 06:55:24.932755 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260812 06:55:24.932907 28287 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:24.926824 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.52.65:51647 (local address 127.19.52.66:34045)
0812 06:55:24.927010 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0812 06:55:24.927016 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 06:55:24.927250 (+ 234us) server_negotiation.cc:407] Connection header received
0812 06:55:24.927428 (+ 178us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:55:24.927431 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:55:24.927485 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:55:24.927588 (+ 103us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 06:55:24.928173 (+ 585us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:24.929103 (+ 930us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:24.931364 (+ 2261us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:24.932101 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:24.932255 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:55:24.932314 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0812 06:55:24.932670 (+ 356us) server_negotiation.cc:299] Negotiation successful
0812 06:55:24.932755 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":87}
I20260812 06:55:24.933075 28300 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:24.926697 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.66:34045 (local address 127.19.52.65:51647)
0812 06:55:24.927158 (+ 461us) negotiation.cc:107] Waiting for socket to connect
0812 06:55:24.927174 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0812 06:55:24.927308 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:55:24.927650 (+ 342us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:55:24.927653 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:55:24.927682 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 06:55:24.928047 (+ 365us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:55:24.928057 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:55:24.929252 (+ 1195us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:55:24.929256 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:55:24.931163 (+ 1907us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:55:24.931177 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:55:24.932280 (+ 1103us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:55:24.932285 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:55:24.932464 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:55:24.932502 (+ 38us) client_negotiation.cc:769] Sending connection context
0812 06:55:24.932647 (+ 145us) client_negotiation.cc:240] Negotiation successful
0812 06:55:24.932806 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":370,"mutex_wait_us":406,"spinlock_wait_cycles":3456,"thread_start_us":147,"threads_started":1}
I20260812 06:55:24.933563 28301 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:24.926832 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.67:45705 (local address 127.19.52.65:48881)
0812 06:55:24.927202 (+ 370us) negotiation.cc:107] Waiting for socket to connect
0812 06:55:24.927217 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 06:55:24.927316 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:55:24.927650 (+ 334us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:55:24.927653 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:55:24.927680 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 06:55:24.928004 (+ 324us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:55:24.928015 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:55:24.929252 (+ 1237us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:55:24.929256 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:55:24.931163 (+ 1907us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:55:24.931177 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:55:24.932280 (+ 1103us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:55:24.932285 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:55:24.932522 (+ 237us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:55:24.932583 (+ 61us) client_negotiation.cc:769] Sending connection context
0812 06:55:24.932664 (+ 81us) client_negotiation.cc:240] Negotiation successful
0812 06:55:24.932806 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":300,"mutex_wait_us":397,"spinlock_wait_cycles":7680,"thread_start_us":118,"threads_started":1}
I20260812 06:55:24.933871 28030 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b4ccea11ed5e4cc4ad6f007cb4ca467b" candidate_uuid: "49e259bbf4094665b4e899332664889c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2e97338a875f4bada5bf4e6301558f78" is_pre_election: true
I20260812 06:55:24.934213 28191 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b4ccea11ed5e4cc4ad6f007cb4ca467b" candidate_uuid: "49e259bbf4094665b4e899332664889c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fff2f0f0327b47368c203f7f1aab31fd" is_pre_election: true
I20260812 06:55:24.934326 28030 raft_consensus.cc:2454] T b4ccea11ed5e4cc4ad6f007cb4ca467b P 2e97338a875f4bada5bf4e6301558f78 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49e259bbf4094665b4e899332664889c in term 0.
I20260812 06:55:24.934754 28191 raft_consensus.cc:2454] T b4ccea11ed5e4cc4ad6f007cb4ca467b P fff2f0f0327b47368c203f7f1aab31fd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49e259bbf4094665b4e899332664889c in term 0.
I20260812 06:55:24.935256 27791 leader_election.cc:302] T b4ccea11ed5e4cc4ad6f007cb4ca467b P 49e259bbf4094665b4e899332664889c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 2e97338a875f4bada5bf4e6301558f78, 49e259bbf4094665b4e899332664889c, fff2f0f0327b47368c203f7f1aab31fd; no voters:
I20260812 06:55:24.935609 28298 raft_consensus.cc:2781] T b4ccea11ed5e4cc4ad6f007cb4ca467b P 49e259bbf4094665b4e899332664889c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 06:55:24.935712 28298 raft_consensus.cc:482] T b4ccea11ed5e4cc4ad6f007cb4ca467b P 49e259bbf4094665b4e899332664889c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 06:55:24.935757 28298 raft_consensus.cc:3037] T b4ccea11ed5e4cc4ad6f007cb4ca467b P 49e259bbf4094665b4e899332664889c [term 0 FOLLOWER]: Advancing to term 1
I20260812 06:55:24.937036 28298 raft_consensus.cc:504] T b4ccea11ed5e4cc4ad6f007cb4ca467b P 49e259bbf4094665b4e899332664889c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e97338a875f4bada5bf4e6301558f78" member_type: VOTER last_known_addr { host: "127.19.52.66" port: 34045 } } peers { permanent_uuid: "49e259bbf4094665b4e899332664889c" member_type: VOTER last_known_addr { host: "127.19.52.65" port: 41417 } } peers { permanent_uuid: "fff2f0f0327b47368c203f7f1aab31fd" member_type: VOTER last_known_addr { host: "127.19.52.67" port: 45705 } }
I20260812 06:55:24.937592 28298 leader_election.cc:288] T b4ccea11ed5e4cc4ad6f007cb4ca467b P 49e259bbf4094665b4e899332664889c [CANDIDATE]: Term 1 election: Requested vote from peers 2e97338a875f4bada5bf4e6301558f78 (127.19.52.66:34045), fff2f0f0327b47368c203f7f1aab31fd (127.19.52.67:45705)
I20260812 06:55:24.938129 28191 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b4ccea11ed5e4cc4ad6f007cb4ca467b" candidate_uuid: "49e259bbf4094665b4e899332664889c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fff2f0f0327b47368c203f7f1aab31fd"
I20260812 06:55:24.938172 28030 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b4ccea11ed5e4cc4ad6f007cb4ca467b" candidate_uuid: "49e259bbf4094665b4e899332664889c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2e97338a875f4bada5bf4e6301558f78"
I20260812 06:55:24.938297 28191 raft_consensus.cc:3037] T b4ccea11ed5e4cc4ad6f007cb4ca467b P fff2f0f0327b47368c203f7f1aab31fd [term 0 FOLLOWER]: Advancing to term 1
I20260812 06:55:24.938314 28030 raft_consensus.cc:3037] T b4ccea11ed5e4cc4ad6f007cb4ca467b P 2e97338a875f4bada5bf4e6301558f78 [term 0 FOLLOWER]: Advancing to term 1
I20260812 06:55:24.939633 28030 raft_consensus.cc:2454] T b4ccea11ed5e4cc4ad6f007cb4ca467b P 2e97338a875f4bada5bf4e6301558f78 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49e259bbf4094665b4e899332664889c in term 1.
I20260812 06:55:24.940013 28191 raft_consensus.cc:2454] T b4ccea11ed5e4cc4ad6f007cb4ca467b P fff2f0f0327b47368c203f7f1aab31fd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49e259bbf4094665b4e899332664889c in term 1.
I20260812 06:55:24.940053 27791 leader_election.cc:302] T b4ccea11ed5e4cc4ad6f007cb4ca467b P 49e259bbf4094665b4e899332664889c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2e97338a875f4bada5bf4e6301558f78, 49e259bbf4094665b4e899332664889c; no voters:
I20260812 06:55:24.940523 28298 raft_consensus.cc:2781] T b4ccea11ed5e4cc4ad6f007cb4ca467b P 49e259bbf4094665b4e899332664889c [term 1 FOLLOWER]: Leader election won for term 1
I20260812 06:55:24.940887 28298 raft_consensus.cc:686] T b4ccea11ed5e4cc4ad6f007cb4ca467b P 49e259bbf4094665b4e899332664889c [term 1 LEADER]: Becoming Leader. State: Replica: 49e259bbf4094665b4e899332664889c, State: Running, Role: LEADER
I20260812 06:55:24.941282 28298 consensus_queue.cc:237] T b4ccea11ed5e4cc4ad6f007cb4ca467b P 49e259bbf4094665b4e899332664889c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e97338a875f4bada5bf4e6301558f78" member_type: VOTER last_known_addr { host: "127.19.52.66" port: 34045 } } peers { permanent_uuid: "49e259bbf4094665b4e899332664889c" member_type: VOTER last_known_addr { host: "127.19.52.65" port: 41417 } } peers { permanent_uuid: "fff2f0f0327b47368c203f7f1aab31fd" member_type: VOTER last_known_addr { host: "127.19.52.67" port: 45705 } }
I20260812 06:55:24.945561 26803 catalog_manager.cc:5495] T b4ccea11ed5e4cc4ad6f007cb4ca467b P 49e259bbf4094665b4e899332664889c reported cstate change: term changed from 0 to 1, leader changed from <none> to 49e259bbf4094665b4e899332664889c (127.19.52.65). New cstate: current_term: 1 leader_uuid: "49e259bbf4094665b4e899332664889c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e97338a875f4bada5bf4e6301558f78" member_type: VOTER last_known_addr { host: "127.19.52.66" port: 34045 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "49e259bbf4094665b4e899332664889c" member_type: VOTER last_known_addr { host: "127.19.52.65" port: 41417 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fff2f0f0327b47368c203f7f1aab31fd" member_type: VOTER last_known_addr { host: "127.19.52.67" port: 45705 } health_report { overall_health: UNKNOWN } } }
I20260812 06:55:24.974325 28289 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:24.969305 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36768 (local address 127.19.52.65:41417)
0812 06:55:24.969476 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0812 06:55:24.969480 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 06:55:24.969497 (+ 17us) server_negotiation.cc:407] Connection header received
0812 06:55:24.969615 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:55:24.969617 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:55:24.969695 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:55:24.969815 (+ 120us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 06:55:24.970626 (+ 811us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:24.971217 (+ 591us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:24.972402 (+ 1185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:24.972699 (+ 297us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:24.972807 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:55:24.973278 (+ 471us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 06:55:24.973793 (+ 515us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 06:55:24.973894 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0812 06:55:24.974089 (+ 195us) server_negotiation.cc:299] Negotiation successful
0812 06:55:24.974198 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260812 06:55:24.983803 28030 raft_consensus.cc:1261] T b4ccea11ed5e4cc4ad6f007cb4ca467b P 2e97338a875f4bada5bf4e6301558f78 [term 1 FOLLOWER]: Refusing update from remote peer 49e259bbf4094665b4e899332664889c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 06:55:24.983803 28191 raft_consensus.cc:1261] T b4ccea11ed5e4cc4ad6f007cb4ca467b P fff2f0f0327b47368c203f7f1aab31fd [term 1 FOLLOWER]: Refusing update from remote peer 49e259bbf4094665b4e899332664889c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 06:55:24.984707 28298 consensus_queue.cc:1035] T b4ccea11ed5e4cc4ad6f007cb4ca467b P 49e259bbf4094665b4e899332664889c [LEADER]: Connected to new peer: Peer: permanent_uuid: "2e97338a875f4bada5bf4e6301558f78" member_type: VOTER last_known_addr { host: "127.19.52.66" port: 34045 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 06:55:24.984977 28303 consensus_queue.cc:1035] T b4ccea11ed5e4cc4ad6f007cb4ca467b P 49e259bbf4094665b4e899332664889c [LEADER]: Connected to new peer: Peer: permanent_uuid: "fff2f0f0327b47368c203f7f1aab31fd" member_type: VOTER last_known_addr { host: "127.19.52.67" port: 45705 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 06:55:24.995052 28309 mvcc.cc:205] Tried to move back new op lower bound from 7317576601524617216 to 7317576601362841600. Current Snapshot: MvccSnapshot[applied={T|T < 7317576601524617216}]
I20260812 06:55:24.997175 28308 mvcc.cc:205] Tried to move back new op lower bound from 7317576601524617216 to 7317576601362841600. Current Snapshot: MvccSnapshot[applied={T|T < 7317576601524617216}]
I20260812 06:55:24.999801 28307 mvcc.cc:205] Tried to move back new op lower bound from 7317576601524617216 to 7317576601362841600. Current Snapshot: MvccSnapshot[applied={T|T < 7317576601524617216 or (T in {7317576601524617216})}]
I20260812 06:55:25.008975 26802 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44494:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 06:55:25.009217 26802 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44494:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 06:55:25.015841 26802 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 53e6ebc3b8ee4672983fb82b50148e26: Sending DeleteTablet for 3 replicas of tablet b4ccea11ed5e4cc4ad6f007cb4ca467b
I20260812 06:55:25.017064 28011 tablet_service.cc:1514] Processing DeleteTablet for tablet b4ccea11ed5e4cc4ad6f007cb4ca467b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 06:55:25 UTC) from {username='oryx', principal='oryx/127.19.52.126@KRBTEST.COM'} at 127.0.0.1:43876
I20260812 06:55:25.017016 27843 tablet_service.cc:1514] Processing DeleteTablet for tablet b4ccea11ed5e4cc4ad6f007cb4ca467b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 06:55:25 UTC) from {username='oryx', principal='oryx/127.19.52.126@KRBTEST.COM'} at 127.0.0.1:36754
I20260812 06:55:25.017393 28167 tablet_service.cc:1514] Processing DeleteTablet for tablet b4ccea11ed5e4cc4ad6f007cb4ca467b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 06:55:25 UTC) from {username='oryx', principal='oryx/127.19.52.126@KRBTEST.COM'} at 127.0.0.1:33194
I20260812 06:55:25.017679 28317 tablet_replica.cc:330] T b4ccea11ed5e4cc4ad6f007cb4ca467b P 2e97338a875f4bada5bf4e6301558f78: stopping tablet replica
I20260812 06:55:25.017925 28316 tablet_replica.cc:330] T b4ccea11ed5e4cc4ad6f007cb4ca467b P 49e259bbf4094665b4e899332664889c: stopping tablet replica
I20260812 06:55:25.018107 28317 raft_consensus.cc:2229] T b4ccea11ed5e4cc4ad6f007cb4ca467b P 2e97338a875f4bada5bf4e6301558f78 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 06:55:25.018347 28316 raft_consensus.cc:2229] T b4ccea11ed5e4cc4ad6f007cb4ca467b P 49e259bbf4094665b4e899332664889c [term 1 LEADER]: Raft consensus shutting down.
I20260812 06:55:25.018498 28317 raft_consensus.cc:2258] T b4ccea11ed5e4cc4ad6f007cb4ca467b P 2e97338a875f4bada5bf4e6301558f78 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 06:55:25.019462 28316 raft_consensus.cc:2258] T b4ccea11ed5e4cc4ad6f007cb4ca467b P 49e259bbf4094665b4e899332664889c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 06:55:25.020213 28317 ts_tablet_manager.cc:1905] T b4ccea11ed5e4cc4ad6f007cb4ca467b P 2e97338a875f4bada5bf4e6301558f78: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 06:55:25.021200 28316 ts_tablet_manager.cc:1905] T b4ccea11ed5e4cc4ad6f007cb4ca467b P 49e259bbf4094665b4e899332664889c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 06:55:25.021477 19665 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu with pid 26768
I20260812 06:55:25.027356 28317 ts_tablet_manager.cc:1918] T b4ccea11ed5e4cc4ad6f007cb4ca467b P 2e97338a875f4bada5bf4e6301558f78: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 06:55:25.027520 28317 log.cc:1196] T b4ccea11ed5e4cc4ad6f007cb4ca467b P 2e97338a875f4bada5bf4e6301558f78: Deleting WAL directory at /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517687384214-19665-0/minicluster-data/ts-1/wal/wals/b4ccea11ed5e4cc4ad6f007cb4ca467b
I20260812 06:55:25.027907 28317 ts_tablet_manager.cc:1939] T b4ccea11ed5e4cc4ad6f007cb4ca467b P 2e97338a875f4bada5bf4e6301558f78: Deleting consensus metadata
I20260812 06:55:25.028887 28316 ts_tablet_manager.cc:1918] T b4ccea11ed5e4cc4ad6f007cb4ca467b P 49e259bbf4094665b4e899332664889c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 06:55:25.029145 28316 log.cc:1196] T b4ccea11ed5e4cc4ad6f007cb4ca467b P 49e259bbf4094665b4e899332664889c: Deleting WAL directory at /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517687384214-19665-0/minicluster-data/ts-0/wal/wals/b4ccea11ed5e4cc4ad6f007cb4ca467b
I20260812 06:55:25.029596 28316 ts_tablet_manager.cc:1939] T b4ccea11ed5e4cc4ad6f007cb4ca467b P 49e259bbf4094665b4e899332664889c: Deleting consensus metadata
I20260812 06:55:25.030079 28318 tablet_replica.cc:330] T b4ccea11ed5e4cc4ad6f007cb4ca467b P fff2f0f0327b47368c203f7f1aab31fd: stopping tablet replica
I20260812 06:55:25.031687 28318 raft_consensus.cc:2229] T b4ccea11ed5e4cc4ad6f007cb4ca467b P fff2f0f0327b47368c203f7f1aab31fd [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 06:55:25.032128 28318 raft_consensus.cc:2258] T b4ccea11ed5e4cc4ad6f007cb4ca467b P fff2f0f0327b47368c203f7f1aab31fd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 06:55:25.034682 28318 ts_tablet_manager.cc:1905] T b4ccea11ed5e4cc4ad6f007cb4ca467b P fff2f0f0327b47368c203f7f1aab31fd: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260812 06:55:25.036448 27788 connection.cc:535] server connection from 127.0.0.1:36754 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260812 06:55:25.037127 27949 connection.cc:535] server connection from 127.0.0.1:43876 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260812 06:55:25.038637 28318 ts_tablet_manager.cc:1918] T b4ccea11ed5e4cc4ad6f007cb4ca467b P fff2f0f0327b47368c203f7f1aab31fd: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 06:55:25.038800 28318 log.cc:1196] T b4ccea11ed5e4cc4ad6f007cb4ca467b P fff2f0f0327b47368c203f7f1aab31fd: Deleting WAL directory at /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517687384214-19665-0/minicluster-data/ts-2/wal/wals/b4ccea11ed5e4cc4ad6f007cb4ca467b
I20260812 06:55:25.039393 28318 ts_tablet_manager.cc:1939] T b4ccea11ed5e4cc4ad6f007cb4ca467b P fff2f0f0327b47368c203f7f1aab31fd: Deleting consensus metadata
W20260812 06:55:25.041432 28116 connection.cc:441] server connection from 127.0.0.1:33194 torn down before Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:33194 (request call id 1) could send its response
I20260812 06:55:25.048640 28259 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:25.019747 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55290 (local address 127.19.52.125:32823)
0812 06:55:25.020016 (+ 269us) server_negotiation.cc:206] Beginning negotiation
0812 06:55:25.020021 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 06:55:25.029814 (+ 9793us) server_negotiation.cc:407] Connection header received
0812 06:55:25.030077 (+ 263us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:55:25.030080 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:55:25.030145 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:55:25.030206 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:55:25.031310 (+ 1104us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:25.032082 (+ 772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:25.033542 (+ 1460us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:25.033968 (+ 426us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:25.034067 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:55:25.043405 (+ 9338us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:55:25.043409 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:55:25.043412 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:55:25.043457 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:55:25.045529 (+ 2072us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:55:25.046080 (+ 551us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:55:25.046084 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:55:25.046086 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:55:25.046224 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:55:25.046610 (+ 386us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:55:25.046613 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:55:25.046615 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:55:25.046837 (+ 222us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:55:25.046959 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0812 06:55:25.047909 (+ 950us) server_negotiation.cc:299] Negotiation successful
0812 06:55:25.048052 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":179}
I20260812 06:55:25.054185 27645 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:25.042911 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55298 (local address 127.19.52.125:32823)
0812 06:55:25.043086 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0812 06:55:25.043091 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 06:55:25.043382 (+ 291us) server_negotiation.cc:407] Connection header received
0812 06:55:25.043496 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:55:25.043500 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:55:25.043562 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:55:25.043654 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:55:25.044532 (+ 878us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:25.045484 (+ 952us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:25.047390 (+ 1906us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:25.047779 (+ 389us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:25.047888 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:55:25.050052 (+ 2164us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:55:25.050056 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:55:25.050058 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:55:25.050091 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:55:25.051924 (+ 1833us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:55:25.052531 (+ 607us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:55:25.052535 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:55:25.052536 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:55:25.052694 (+ 158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:55:25.053033 (+ 339us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:55:25.053037 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:55:25.053038 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:55:25.053257 (+ 219us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:55:25.053370 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0812 06:55:25.053876 (+ 506us) server_negotiation.cc:299] Negotiation successful
0812 06:55:25.054010 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260812 06:55:25.060076 28264 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:25.019875 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36328 (local address 127.19.52.124:45647)
0812 06:55:25.020118 (+ 243us) server_negotiation.cc:206] Beginning negotiation
0812 06:55:25.020122 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 06:55:25.048119 (+ 27997us) server_negotiation.cc:407] Connection header received
0812 06:55:25.048824 (+ 705us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:55:25.048828 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:55:25.048886 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:55:25.048938 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:55:25.049840 (+ 902us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:25.050666 (+ 826us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:25.052035 (+ 1369us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:25.052627 (+ 592us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:25.052752 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:55:25.055851 (+ 3099us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:55:25.055855 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:55:25.055858 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:55:25.055887 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:55:25.057770 (+ 1883us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:55:25.058268 (+ 498us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:55:25.058272 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:55:25.058275 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:55:25.058380 (+ 105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:55:25.058962 (+ 582us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:55:25.058966 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:55:25.058968 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:55:25.059205 (+ 237us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:55:25.059354 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0812 06:55:25.059713 (+ 359us) server_negotiation.cc:299] Negotiation successful
0812 06:55:25.059878 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":165}
I20260812 06:55:25.062569 27927 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:25.045712 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36338 (local address 127.19.52.124:45647)
0812 06:55:25.045857 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0812 06:55:25.045862 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 06:55:25.051018 (+ 5156us) server_negotiation.cc:407] Connection header received
0812 06:55:25.051197 (+ 179us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:55:25.051200 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:55:25.051249 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:55:25.051297 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:55:25.052746 (+ 1449us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:25.053766 (+ 1020us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:25.055607 (+ 1841us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:25.055962 (+ 355us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:25.056044 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:55:25.058167 (+ 2123us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:55:25.058173 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:55:25.058179 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:55:25.058215 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:55:25.060594 (+ 2379us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:55:25.061101 (+ 507us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:55:25.061105 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:55:25.061107 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:55:25.061194 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:55:25.061556 (+ 362us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:55:25.061560 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:55:25.061562 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:55:25.061792 (+ 230us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:55:25.061959 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0812 06:55:25.062263 (+ 304us) server_negotiation.cc:299] Negotiation successful
0812 06:55:25.062391 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":120,"server-negotiator.queue_time_us":66}
I20260812 06:55:26.000816 28326 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:26.000218 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.126:34933 (local address 127.19.52.67:47335)
0812 06:55:26.000496 (+ 278us) negotiation.cc:107] Waiting for socket to connect
0812 06:55:26.000658 (+ 162us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.52.126:34933: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":187,"thread_start_us":104,"threads_started":1}
W20260812 06:55:26.001271 28244 heartbeater.cc:644] Failed to heartbeat to 127.19.52.126:34933 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.19.52.126:34933: connect: Connection refused (error 111)
I20260812 06:55:26.003468 28327 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:26.003003 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.126:34933 (local address 127.19.52.66:58085)
0812 06:55:26.003303 (+ 300us) negotiation.cc:107] Waiting for socket to connect
0812 06:55:26.003381 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.52.126:34933: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":224,"thread_start_us":102,"threads_started":1}
W20260812 06:55:26.003849 28086 heartbeater.cc:644] Failed to heartbeat to 127.19.52.126:34933 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.19.52.126:34933: connect: Connection refused (error 111)
I20260812 06:55:26.009006 28329 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:26.008459 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.126:34933 (local address 127.19.52.65:44157)
0812 06:55:26.008813 (+ 354us) negotiation.cc:107] Waiting for socket to connect
0812 06:55:26.008907 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.52.126:34933: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":233,"thread_start_us":117,"threads_started":1}
W20260812 06:55:26.009474 27924 heartbeater.cc:644] Failed to heartbeat to 127.19.52.126:34933 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.19.52.126:34933: connect: Connection refused (error 111)
I20260812 06:55:26.515625 28330 raft_consensus.cc:482] T 00000000000000000000000000000000 P 950b5d35439b4e23bd6d86c6664ddc43 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 53e6ebc3b8ee4672983fb82b50148e26)
I20260812 06:55:26.515622 28331 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2b5a510eb7be4cfab2f7914ebabaa929 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 53e6ebc3b8ee4672983fb82b50148e26)
I20260812 06:55:26.515846 28330 raft_consensus.cc:504] T 00000000000000000000000000000000 P 950b5d35439b4e23bd6d86c6664ddc43 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "53e6ebc3b8ee4672983fb82b50148e26" member_type: VOTER last_known_addr { host: "127.19.52.126" port: 34933 } } peers { permanent_uuid: "2b5a510eb7be4cfab2f7914ebabaa929" member_type: VOTER last_known_addr { host: "127.19.52.125" port: 32823 } attrs { promote: false } } peers { permanent_uuid: "950b5d35439b4e23bd6d86c6664ddc43" member_type: VOTER last_known_addr { host: "127.19.52.124" port: 45647 } attrs { promote: false } }
I20260812 06:55:26.515887 28331 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2b5a510eb7be4cfab2f7914ebabaa929 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "53e6ebc3b8ee4672983fb82b50148e26" member_type: VOTER last_known_addr { host: "127.19.52.126" port: 34933 } } peers { permanent_uuid: "2b5a510eb7be4cfab2f7914ebabaa929" member_type: VOTER last_known_addr { host: "127.19.52.125" port: 32823 } attrs { promote: false } } peers { permanent_uuid: "950b5d35439b4e23bd6d86c6664ddc43" member_type: VOTER last_known_addr { host: "127.19.52.124" port: 45647 } attrs { promote: false } }
I20260812 06:55:26.517068 28331 leader_election.cc:288] T 00000000000000000000000000000000 P 2b5a510eb7be4cfab2f7914ebabaa929 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 53e6ebc3b8ee4672983fb82b50148e26 (127.19.52.126:34933), 950b5d35439b4e23bd6d86c6664ddc43 (127.19.52.124:45647)
I20260812 06:55:26.517623 28330 leader_election.cc:288] T 00000000000000000000000000000000 P 950b5d35439b4e23bd6d86c6664ddc43 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 53e6ebc3b8ee4672983fb82b50148e26 (127.19.52.126:34933), 2b5a510eb7be4cfab2f7914ebabaa929 (127.19.52.125:32823)
I20260812 06:55:26.517781 28332 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:26.517194 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.126:34933 (local address 127.0.0.1:51740)
0812 06:55:26.517590 (+ 396us) negotiation.cc:107] Waiting for socket to connect
0812 06:55:26.517673 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.52.126:34933: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":314,"thread_start_us":72,"threads_started":1}
W20260812 06:55:26.517949 27484 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.19.52.126:34933: connect: Connection refused (error 111)
I20260812 06:55:26.518333 28334 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:26.517205 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.126:34933 (local address 127.0.0.1:51738)
0812 06:55:26.518144 (+ 939us) negotiation.cc:107] Waiting for socket to connect
0812 06:55:26.518219 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.52.126:34933: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":876,"thread_start_us":72,"threads_started":1}
W20260812 06:55:26.518621 27698 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.19.52.126:34933: connect: Connection refused (error 111)
I20260812 06:55:26.518906 28336 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:26.518569 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.126:34933 (local address 127.0.0.1:51762)
0812 06:55:26.518746 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0812 06:55:26.518816 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.52.126:34933: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
W20260812 06:55:26.519083 27484 leader_election.cc:334] T 00000000000000000000000000000000 P 2b5a510eb7be4cfab2f7914ebabaa929 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 53e6ebc3b8ee4672983fb82b50148e26 (127.19.52.126:34933): Network error: Client connection negotiation failed: client connection to 127.19.52.126:34933: connect: Connection refused (error 111)
I20260812 06:55:26.519812 28334 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:26.519581 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.126:34933 (local address 127.0.0.1:51764)
0812 06:55:26.519703 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0812 06:55:26.519754 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.52.126:34933: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260812 06:55:26.520016 27698 leader_election.cc:334] T 00000000000000000000000000000000 P 950b5d35439b4e23bd6d86c6664ddc43 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 53e6ebc3b8ee4672983fb82b50148e26 (127.19.52.126:34933): Network error: Client connection negotiation failed: client connection to 127.19.52.126:34933: connect: Connection refused (error 111)
I20260812 06:55:26.522107 28337 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:26.517486 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52598 (local address 127.19.52.125:32823)
0812 06:55:26.518333 (+ 847us) server_negotiation.cc:206] Beginning negotiation
0812 06:55:26.518341 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 06:55:26.518356 (+ 15us) server_negotiation.cc:407] Connection header received
0812 06:55:26.518399 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:55:26.518402 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:55:26.518450 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:55:26.518844 (+ 394us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 06:55:26.519138 (+ 294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:26.519842 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:26.521051 (+ 1209us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:26.521527 (+ 476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:26.521637 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:55:26.521698 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0812 06:55:26.521911 (+ 213us) server_negotiation.cc:299] Negotiation successful
0812 06:55:26.521986 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":284,"spinlock_wait_cycles":386304,"thread_start_us":48,"threads_started":1}
I20260812 06:55:26.522125 28333 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:26.517194 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.125:32823 (local address 127.0.0.1:52598)
0812 06:55:26.517543 (+ 349us) negotiation.cc:107] Waiting for socket to connect
0812 06:55:26.517561 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0812 06:55:26.517682 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:55:26.518586 (+ 904us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:55:26.518590 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:55:26.518622 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 06:55:26.518732 (+ 110us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:55:26.518742 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:55:26.519980 (+ 1238us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:55:26.519986 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:55:26.520924 (+ 938us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:55:26.520933 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:55:26.521633 (+ 700us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:55:26.521637 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:55:26.521769 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:55:26.521798 (+ 29us) client_negotiation.cc:769] Sending connection context
0812 06:55:26.521884 (+ 86us) client_negotiation.cc:240] Negotiation successful
0812 06:55:26.521964 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":247,"mutex_wait_us":3,"thread_start_us":69,"threads_started":1}
I20260812 06:55:26.522732 28332 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:26.517427 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.124:45647 (local address 127.0.0.1:48984)
0812 06:55:26.517911 (+ 484us) negotiation.cc:107] Waiting for socket to connect
0812 06:55:26.517924 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 06:55:26.518019 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:55:26.518477 (+ 458us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:55:26.518484 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:55:26.519003 (+ 519us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 06:55:26.519138 (+ 135us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:55:26.519145 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:55:26.520231 (+ 1086us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:55:26.520234 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:55:26.521183 (+ 949us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:55:26.521193 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:55:26.522298 (+ 1105us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:55:26.522302 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:55:26.522419 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:55:26.522444 (+ 25us) client_negotiation.cc:769] Sending connection context
0812 06:55:26.522516 (+ 72us) client_negotiation.cc:240] Negotiation successful
0812 06:55:26.522593 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":442,"spinlock_wait_cycles":7424,"thread_start_us":68,"threads_started":1}
I20260812 06:55:26.522809 27514 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "950b5d35439b4e23bd6d86c6664ddc43" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "2b5a510eb7be4cfab2f7914ebabaa929" is_pre_election: true
I20260812 06:55:26.523052 27514 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 2b5a510eb7be4cfab2f7914ebabaa929 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 950b5d35439b4e23bd6d86c6664ddc43 in term 1.
I20260812 06:55:26.523202 28338 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:26.517853 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48984 (local address 127.19.52.124:45647)
0812 06:55:26.518083 (+ 230us) server_negotiation.cc:206] Beginning negotiation
0812 06:55:26.518088 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 06:55:26.518104 (+ 16us) server_negotiation.cc:407] Connection header received
0812 06:55:26.518154 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:55:26.518157 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:55:26.518210 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:55:26.518315 (+ 105us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 06:55:26.519320 (+ 1005us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:26.520110 (+ 790us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:26.521320 (+ 1210us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:26.522045 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:26.522158 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:55:26.522225 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0812 06:55:26.522973 (+ 748us) server_negotiation.cc:299] Negotiation successful
0812 06:55:26.523075 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":47,"server-negotiator.queue_time_us":153,"thread_start_us":57,"threads_started":1}
I20260812 06:55:26.523504 27699 leader_election.cc:302] T 00000000000000000000000000000000 P 950b5d35439b4e23bd6d86c6664ddc43 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2b5a510eb7be4cfab2f7914ebabaa929, 950b5d35439b4e23bd6d86c6664ddc43; no voters: 53e6ebc3b8ee4672983fb82b50148e26
I20260812 06:55:26.523577 27723 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "2b5a510eb7be4cfab2f7914ebabaa929" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "950b5d35439b4e23bd6d86c6664ddc43" is_pre_election: true
I20260812 06:55:26.523758 27723 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 950b5d35439b4e23bd6d86c6664ddc43 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2b5a510eb7be4cfab2f7914ebabaa929 in term 1.
I20260812 06:55:26.523842 28330 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 950b5d35439b4e23bd6d86c6664ddc43 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260812 06:55:26.523904 28330 raft_consensus.cc:482] T 00000000000000000000000000000000 P 950b5d35439b4e23bd6d86c6664ddc43 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 53e6ebc3b8ee4672983fb82b50148e26)
I20260812 06:55:26.523943 28330 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 950b5d35439b4e23bd6d86c6664ddc43 [term 1 FOLLOWER]: Advancing to term 2
I20260812 06:55:26.524080 27483 leader_election.cc:302] T 00000000000000000000000000000000 P 2b5a510eb7be4cfab2f7914ebabaa929 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2b5a510eb7be4cfab2f7914ebabaa929, 950b5d35439b4e23bd6d86c6664ddc43; no voters: 53e6ebc3b8ee4672983fb82b50148e26
I20260812 06:55:26.524286 28331 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2b5a510eb7be4cfab2f7914ebabaa929 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260812 06:55:26.524338 28331 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2b5a510eb7be4cfab2f7914ebabaa929 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 53e6ebc3b8ee4672983fb82b50148e26)
I20260812 06:55:26.524381 28331 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2b5a510eb7be4cfab2f7914ebabaa929 [term 1 FOLLOWER]: Advancing to term 2
I20260812 06:55:26.525134 28330 raft_consensus.cc:504] T 00000000000000000000000000000000 P 950b5d35439b4e23bd6d86c6664ddc43 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "53e6ebc3b8ee4672983fb82b50148e26" member_type: VOTER last_known_addr { host: "127.19.52.126" port: 34933 } } peers { permanent_uuid: "2b5a510eb7be4cfab2f7914ebabaa929" member_type: VOTER last_known_addr { host: "127.19.52.125" port: 32823 } attrs { promote: false } } peers { permanent_uuid: "950b5d35439b4e23bd6d86c6664ddc43" member_type: VOTER last_known_addr { host: "127.19.52.124" port: 45647 } attrs { promote: false } }
I20260812 06:55:26.525575 28330 leader_election.cc:288] T 00000000000000000000000000000000 P 950b5d35439b4e23bd6d86c6664ddc43 [CANDIDATE]: Term 2 election: Requested vote from peers 53e6ebc3b8ee4672983fb82b50148e26 (127.19.52.126:34933), 2b5a510eb7be4cfab2f7914ebabaa929 (127.19.52.125:32823)
I20260812 06:55:26.525672 28331 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2b5a510eb7be4cfab2f7914ebabaa929 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "53e6ebc3b8ee4672983fb82b50148e26" member_type: VOTER last_known_addr { host: "127.19.52.126" port: 34933 } } peers { permanent_uuid: "2b5a510eb7be4cfab2f7914ebabaa929" member_type: VOTER last_known_addr { host: "127.19.52.125" port: 32823 } attrs { promote: false } } peers { permanent_uuid: "950b5d35439b4e23bd6d86c6664ddc43" member_type: VOTER last_known_addr { host: "127.19.52.124" port: 45647 } attrs { promote: false } }
I20260812 06:55:26.525954 28333 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:26.525719 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.126:34933 (local address 127.0.0.1:51780)
0812 06:55:26.525841 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0812 06:55:26.525906 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.52.126:34933: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
I20260812 06:55:26.525957 27514 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "950b5d35439b4e23bd6d86c6664ddc43" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "2b5a510eb7be4cfab2f7914ebabaa929"
I20260812 06:55:26.526126 27514 raft_consensus.cc:2379] T 00000000000000000000000000000000 P 2b5a510eb7be4cfab2f7914ebabaa929 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 950b5d35439b4e23bd6d86c6664ddc43 in current term 2: Already voted for candidate 2b5a510eb7be4cfab2f7914ebabaa929 in this term.
I20260812 06:55:26.526569 27723 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "2b5a510eb7be4cfab2f7914ebabaa929" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "950b5d35439b4e23bd6d86c6664ddc43"
I20260812 06:55:26.526746 27723 raft_consensus.cc:2379] T 00000000000000000000000000000000 P 950b5d35439b4e23bd6d86c6664ddc43 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 2b5a510eb7be4cfab2f7914ebabaa929 in current term 2: Already voted for candidate 950b5d35439b4e23bd6d86c6664ddc43 in this term.
I20260812 06:55:26.526126 28331 leader_election.cc:288] T 00000000000000000000000000000000 P 2b5a510eb7be4cfab2f7914ebabaa929 [CANDIDATE]: Term 2 election: Requested vote from peers 53e6ebc3b8ee4672983fb82b50148e26 (127.19.52.126:34933), 950b5d35439b4e23bd6d86c6664ddc43 (127.19.52.124:45647)
I20260812 06:55:26.526602 28332 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:26.526292 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.126:34933 (local address 127.0.0.1:51782)
0812 06:55:26.526473 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0812 06:55:26.526530 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.52.126:34933: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":96}
I20260812 06:55:26.527397 28333 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:26.527047 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.126:34933 (local address 127.0.0.1:51792)
0812 06:55:26.527243 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0812 06:55:26.527306 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.52.126:34933: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":83}
W20260812 06:55:26.527529 27698 leader_election.cc:334] T 00000000000000000000000000000000 P 950b5d35439b4e23bd6d86c6664ddc43 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 53e6ebc3b8ee4672983fb82b50148e26 (127.19.52.126:34933): Network error: Client connection negotiation failed: client connection to 127.19.52.126:34933: connect: Connection refused (error 111)
I20260812 06:55:26.527616 27698 leader_election.cc:302] T 00000000000000000000000000000000 P 950b5d35439b4e23bd6d86c6664ddc43 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 950b5d35439b4e23bd6d86c6664ddc43; no voters: 2b5a510eb7be4cfab2f7914ebabaa929, 53e6ebc3b8ee4672983fb82b50148e26
I20260812 06:55:26.527742 28330 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 950b5d35439b4e23bd6d86c6664ddc43 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260812 06:55:26.527889 28332 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:26.527596 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.126:34933 (local address 127.0.0.1:51796)
0812 06:55:26.527742 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0812 06:55:26.527826 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.52.126:34933: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":72}
W20260812 06:55:26.528010 27484 leader_election.cc:334] T 00000000000000000000000000000000 P 2b5a510eb7be4cfab2f7914ebabaa929 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 53e6ebc3b8ee4672983fb82b50148e26 (127.19.52.126:34933): Network error: Client connection negotiation failed: client connection to 127.19.52.126:34933: connect: Connection refused (error 111)
I20260812 06:55:26.528139 27484 leader_election.cc:302] T 00000000000000000000000000000000 P 2b5a510eb7be4cfab2f7914ebabaa929 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2b5a510eb7be4cfab2f7914ebabaa929; no voters: 53e6ebc3b8ee4672983fb82b50148e26, 950b5d35439b4e23bd6d86c6664ddc43
I20260812 06:55:26.528299 28331 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 2b5a510eb7be4cfab2f7914ebabaa929 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260812 06:55:27.002691 28341 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:27.002074 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.126:34933 (local address 127.19.52.67:54631)
0812 06:55:27.002450 (+ 376us) negotiation.cc:107] Waiting for socket to connect
0812 06:55:27.002547 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.52.126:34933: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":263,"thread_start_us":156,"threads_started":1}
I20260812 06:55:27.005117 28342 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:27.004401 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.126:34933 (local address 127.19.52.66:60737)
0812 06:55:27.004917 (+ 516us) negotiation.cc:107] Waiting for socket to connect
0812 06:55:27.004995 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.52.126:34933: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":428,"thread_start_us":93,"threads_started":1}
I20260812 06:55:27.010617 28343 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:27.010136 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.126:34933 (local address 127.19.52.65:55651)
0812 06:55:27.010465 (+ 329us) negotiation.cc:107] Waiting for socket to connect
0812 06:55:27.010528 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.52.126:34933: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":226,"thread_start_us":95,"threads_started":1}
I20260812 06:55:28.004401 28344 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:28.003711 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.126:34933 (local address 127.19.52.67:59119)
0812 06:55:28.004177 (+ 466us) negotiation.cc:107] Waiting for socket to connect
0812 06:55:28.004263 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.52.126:34933: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":212,"thread_start_us":88,"threads_started":1}
I20260812 06:55:28.006366 28345 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:28.005943 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.126:34933 (local address 127.19.52.66:49617)
0812 06:55:28.006208 (+ 265us) negotiation.cc:107] Waiting for socket to connect
0812 06:55:28.006277 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.52.126:34933: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":167,"thread_start_us":89,"threads_started":1}
I20260812 06:55:28.012128 28346 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:28.011610 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.126:34933 (local address 127.19.52.65:56263)
0812 06:55:28.011927 (+ 317us) negotiation.cc:107] Waiting for socket to connect
0812 06:55:28.011994 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.52.126:34933: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":237,"thread_start_us":96,"threads_started":1}
I20260812 06:55:28.500703 28347 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2b5a510eb7be4cfab2f7914ebabaa929 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 06:55:28.500998 28347 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2b5a510eb7be4cfab2f7914ebabaa929 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "53e6ebc3b8ee4672983fb82b50148e26" member_type: VOTER last_known_addr { host: "127.19.52.126" port: 34933 } } peers { permanent_uuid: "2b5a510eb7be4cfab2f7914ebabaa929" member_type: VOTER last_known_addr { host: "127.19.52.125" port: 32823 } attrs { promote: false } } peers { permanent_uuid: "950b5d35439b4e23bd6d86c6664ddc43" member_type: VOTER last_known_addr { host: "127.19.52.124" port: 45647 } attrs { promote: false } }
I20260812 06:55:28.501549 28347 leader_election.cc:288] T 00000000000000000000000000000000 P 2b5a510eb7be4cfab2f7914ebabaa929 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 53e6ebc3b8ee4672983fb82b50148e26 (127.19.52.126:34933), 950b5d35439b4e23bd6d86c6664ddc43 (127.19.52.124:45647)
I20260812 06:55:28.502187 27723 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "2b5a510eb7be4cfab2f7914ebabaa929" candidate_term: 3 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "950b5d35439b4e23bd6d86c6664ddc43" is_pre_election: true
I20260812 06:55:28.502235 28348 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:28.501754 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.126:34933 (local address 127.0.0.1:52076)
0812 06:55:28.502068 (+ 314us) negotiation.cc:107] Waiting for socket to connect
0812 06:55:28.502145 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.52.126:34933: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":179,"thread_start_us":66,"threads_started":1}
I20260812 06:55:28.502341 27723 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 950b5d35439b4e23bd6d86c6664ddc43 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2b5a510eb7be4cfab2f7914ebabaa929 in term 2.
I20260812 06:55:28.502722 27483 leader_election.cc:302] T 00000000000000000000000000000000 P 2b5a510eb7be4cfab2f7914ebabaa929 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2b5a510eb7be4cfab2f7914ebabaa929, 950b5d35439b4e23bd6d86c6664ddc43; no voters:
I20260812 06:55:28.502934 28347 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2b5a510eb7be4cfab2f7914ebabaa929 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260812 06:55:28.503005 28347 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2b5a510eb7be4cfab2f7914ebabaa929 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 06:55:28.503068 28347 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2b5a510eb7be4cfab2f7914ebabaa929 [term 2 FOLLOWER]: Advancing to term 3
I20260812 06:55:28.503271 28348 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:28.503046 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.126:34933 (local address 127.0.0.1:52078)
0812 06:55:28.503152 (+ 106us) negotiation.cc:107] Waiting for socket to connect
0812 06:55:28.503212 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.52.126:34933: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
W20260812 06:55:28.503388 27484 leader_election.cc:334] T 00000000000000000000000000000000 P 2b5a510eb7be4cfab2f7914ebabaa929 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 53e6ebc3b8ee4672983fb82b50148e26 (127.19.52.126:34933): Network error: Client connection negotiation failed: client connection to 127.19.52.126:34933: connect: Connection refused (error 111)
I20260812 06:55:28.504422 28347 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2b5a510eb7be4cfab2f7914ebabaa929 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "53e6ebc3b8ee4672983fb82b50148e26" member_type: VOTER last_known_addr { host: "127.19.52.126" port: 34933 } } peers { permanent_uuid: "2b5a510eb7be4cfab2f7914ebabaa929" member_type: VOTER last_known_addr { host: "127.19.52.125" port: 32823 } attrs { promote: false } } peers { permanent_uuid: "950b5d35439b4e23bd6d86c6664ddc43" member_type: VOTER last_known_addr { host: "127.19.52.124" port: 45647 } attrs { promote: false } }
I20260812 06:55:28.504899 28347 leader_election.cc:288] T 00000000000000000000000000000000 P 2b5a510eb7be4cfab2f7914ebabaa929 [CANDIDATE]: Term 3 election: Requested vote from peers 53e6ebc3b8ee4672983fb82b50148e26 (127.19.52.126:34933), 950b5d35439b4e23bd6d86c6664ddc43 (127.19.52.124:45647)
I20260812 06:55:28.505330 28348 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:28.505034 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.126:34933 (local address 127.0.0.1:52090)
0812 06:55:28.505148 (+ 114us) negotiation.cc:107] Waiting for socket to connect
0812 06:55:28.505235 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.52.126:34933: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
I20260812 06:55:28.505354 27723 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "2b5a510eb7be4cfab2f7914ebabaa929" candidate_term: 3 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "950b5d35439b4e23bd6d86c6664ddc43"
I20260812 06:55:28.505537 27723 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 950b5d35439b4e23bd6d86c6664ddc43 [term 2 FOLLOWER]: Advancing to term 3
I20260812 06:55:28.506094 28348 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:28.505860 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.126:34933 (local address 127.0.0.1:52092)
0812 06:55:28.505973 (+ 113us) negotiation.cc:107] Waiting for socket to connect
0812 06:55:28.506027 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.52.126:34933: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260812 06:55:28.506210 27484 leader_election.cc:334] T 00000000000000000000000000000000 P 2b5a510eb7be4cfab2f7914ebabaa929 [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer 53e6ebc3b8ee4672983fb82b50148e26 (127.19.52.126:34933): Network error: Client connection negotiation failed: client connection to 127.19.52.126:34933: connect: Connection refused (error 111)
I20260812 06:55:28.506776 27723 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 950b5d35439b4e23bd6d86c6664ddc43 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2b5a510eb7be4cfab2f7914ebabaa929 in term 3.
I20260812 06:55:28.507184 27483 leader_election.cc:302] T 00000000000000000000000000000000 P 2b5a510eb7be4cfab2f7914ebabaa929 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2b5a510eb7be4cfab2f7914ebabaa929, 950b5d35439b4e23bd6d86c6664ddc43; no voters: 53e6ebc3b8ee4672983fb82b50148e26
I20260812 06:55:28.507369 28347 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2b5a510eb7be4cfab2f7914ebabaa929 [term 3 FOLLOWER]: Leader election won for term 3
I20260812 06:55:28.507599 28347 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2b5a510eb7be4cfab2f7914ebabaa929 [term 3 LEADER]: Becoming Leader. State: Replica: 2b5a510eb7be4cfab2f7914ebabaa929, State: Running, Role: LEADER
I20260812 06:55:28.507880 28347 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2b5a510eb7be4cfab2f7914ebabaa929 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 1.12, Last appended by leader: 12, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "53e6ebc3b8ee4672983fb82b50148e26" member_type: VOTER last_known_addr { host: "127.19.52.126" port: 34933 } } peers { permanent_uuid: "2b5a510eb7be4cfab2f7914ebabaa929" member_type: VOTER last_known_addr { host: "127.19.52.125" port: 32823 } attrs { promote: false } } peers { permanent_uuid: "950b5d35439b4e23bd6d86c6664ddc43" member_type: VOTER last_known_addr { host: "127.19.52.124" port: 45647 } attrs { promote: false } }
I20260812 06:55:28.509487 28350 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2b5a510eb7be4cfab2f7914ebabaa929 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2b5a510eb7be4cfab2f7914ebabaa929. Latest consensus state: current_term: 3 leader_uuid: "2b5a510eb7be4cfab2f7914ebabaa929" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "53e6ebc3b8ee4672983fb82b50148e26" member_type: VOTER last_known_addr { host: "127.19.52.126" port: 34933 } } peers { permanent_uuid: "2b5a510eb7be4cfab2f7914ebabaa929" member_type: VOTER last_known_addr { host: "127.19.52.125" port: 32823 } attrs { promote: false } } peers { permanent_uuid: "950b5d35439b4e23bd6d86c6664ddc43" member_type: VOTER last_known_addr { host: "127.19.52.124" port: 45647 } attrs { promote: false } } }
I20260812 06:55:28.509646 28350 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2b5a510eb7be4cfab2f7914ebabaa929 [sys.catalog]: This master's current role is: LEADER
I20260812 06:55:28.510008 28352 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 06:55:28.512712 28352 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 06:55:28.513116 28352 catalog_manager.cc:1259] Loaded cluster ID: 020dffafdb2c4d38b19668de7f5df860
I20260812 06:55:28.513163 28352 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 06:55:28.513700 28352 catalog_manager.cc:1504] Loading token signing keys...
I20260812 06:55:28.514084 28352 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 2b5a510eb7be4cfab2f7914ebabaa929: Loaded TSK: 0
I20260812 06:55:28.514743 28352 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 06:55:28.538362 27502 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55298:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 06:55:28.541678 27502 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 06:55:28.544893 27723 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 950b5d35439b4e23bd6d86c6664ddc43 [term 3 FOLLOWER]: Refusing update from remote peer 2b5a510eb7be4cfab2f7914ebabaa929: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 3 index: 14. (index mismatch)
I20260812 06:55:28.544930 28348 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:28.544670 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.126:34933 (local address 127.0.0.1:52106)
0812 06:55:28.544800 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0812 06:55:28.544861 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.52.126:34933: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
I20260812 06:55:28.545673 28348 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:28.545396 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.126:34933 (local address 127.0.0.1:52108)
0812 06:55:28.545500 (+ 104us) negotiation.cc:107] Waiting for socket to connect
0812 06:55:28.545558 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.52.126:34933: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
I20260812 06:55:28.545576 28350 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 2b5a510eb7be4cfab2f7914ebabaa929 [LEADER]: Connected to new peer: Peer: permanent_uuid: "950b5d35439b4e23bd6d86c6664ddc43" member_type: VOTER last_known_addr { host: "127.19.52.124" port: 45647 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
W20260812 06:55:28.545819 27484 consensus_peers.cc:487] T 00000000000000000000000000000000 P 2b5a510eb7be4cfab2f7914ebabaa929 -> Peer 53e6ebc3b8ee4672983fb82b50148e26 (127.19.52.126:34933): Couldn't send request to peer 53e6ebc3b8ee4672983fb82b50148e26. Status: Network error: Client connection negotiation failed: client connection to 127.19.52.126:34933: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260812 06:55:28.547122 28354 sys_catalog.cc:455] T 00000000000000000000000000000000 P 950b5d35439b4e23bd6d86c6664ddc43 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2b5a510eb7be4cfab2f7914ebabaa929. Latest consensus state: current_term: 3 leader_uuid: "2b5a510eb7be4cfab2f7914ebabaa929" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "53e6ebc3b8ee4672983fb82b50148e26" member_type: VOTER last_known_addr { host: "127.19.52.126" port: 34933 } } peers { permanent_uuid: "2b5a510eb7be4cfab2f7914ebabaa929" member_type: VOTER last_known_addr { host: "127.19.52.125" port: 32823 } attrs { promote: false } } peers { permanent_uuid: "950b5d35439b4e23bd6d86c6664ddc43" member_type: VOTER last_known_addr { host: "127.19.52.124" port: 45647 } attrs { promote: false } } }
I20260812 06:55:28.547251 28354 sys_catalog.cc:458] T 00000000000000000000000000000000 P 950b5d35439b4e23bd6d86c6664ddc43 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 06:55:28.548338 28350 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2b5a510eb7be4cfab2f7914ebabaa929 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 3 leader_uuid: "2b5a510eb7be4cfab2f7914ebabaa929" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "53e6ebc3b8ee4672983fb82b50148e26" member_type: VOTER last_known_addr { host: "127.19.52.126" port: 34933 } } peers { permanent_uuid: "2b5a510eb7be4cfab2f7914ebabaa929" member_type: VOTER last_known_addr { host: "127.19.52.125" port: 32823 } attrs { promote: false } } peers { permanent_uuid: "950b5d35439b4e23bd6d86c6664ddc43" member_type: VOTER last_known_addr { host: "127.19.52.124" port: 45647 } attrs { promote: false } } }
I20260812 06:55:28.548465 28350 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2b5a510eb7be4cfab2f7914ebabaa929 [sys.catalog]: This master's current role is: LEADER
I20260812 06:55:28.549021 28354 sys_catalog.cc:455] T 00000000000000000000000000000000 P 950b5d35439b4e23bd6d86c6664ddc43 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 3 leader_uuid: "2b5a510eb7be4cfab2f7914ebabaa929" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "53e6ebc3b8ee4672983fb82b50148e26" member_type: VOTER last_known_addr { host: "127.19.52.126" port: 34933 } } peers { permanent_uuid: "2b5a510eb7be4cfab2f7914ebabaa929" member_type: VOTER last_known_addr { host: "127.19.52.125" port: 32823 } attrs { promote: false } } peers { permanent_uuid: "950b5d35439b4e23bd6d86c6664ddc43" member_type: VOTER last_known_addr { host: "127.19.52.124" port: 45647 } attrs { promote: false } } }
I20260812 06:55:28.549139 28354 sys_catalog.cc:458] T 00000000000000000000000000000000 P 950b5d35439b4e23bd6d86c6664ddc43 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 06:55:28.562762 28362 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:28.557335 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54904 (local address 127.19.52.65:41417)
0812 06:55:28.557620 (+ 285us) server_negotiation.cc:206] Beginning negotiation
0812 06:55:28.557624 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 06:55:28.558039 (+ 415us) server_negotiation.cc:407] Connection header received
0812 06:55:28.558207 (+ 168us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:55:28.558215 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:55:28.558260 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:55:28.558364 (+ 104us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 06:55:28.558893 (+ 529us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:28.559615 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:28.561239 (+ 1624us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:28.561786 (+ 547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:28.561915 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:55:28.562044 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0812 06:55:28.562480 (+ 436us) server_negotiation.cc:299] Negotiation successful
0812 06:55:28.562576 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":211,"thread_start_us":94,"threads_started":1}
I20260812 06:55:28.562388 28364 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:28.557202 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.65:41417 (local address 127.0.0.1:54904)
0812 06:55:28.557930 (+ 728us) negotiation.cc:107] Waiting for socket to connect
0812 06:55:28.557945 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 06:55:28.558089 (+ 144us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:55:28.558374 (+ 285us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:55:28.558378 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:55:28.558395 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 06:55:28.558710 (+ 315us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:55:28.558718 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:55:28.559754 (+ 1036us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:55:28.559756 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:55:28.561058 (+ 1302us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:55:28.561071 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:55:28.561921 (+ 850us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:55:28.561924 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:55:28.562009 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:55:28.562036 (+ 27us) client_negotiation.cc:769] Sending connection context
0812 06:55:28.562122 (+ 86us) client_negotiation.cc:240] Negotiation successful
0812 06:55:28.562206 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":666,"mutex_wait_us":20,"thread_start_us":589,"threads_started":1}
I20260812 06:55:28.563607 28348 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:28.556338 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.67:45705 (local address 127.0.0.1:52166)
0812 06:55:28.556465 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0812 06:55:28.556484 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0812 06:55:28.556615 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:55:28.558773 (+ 2158us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:55:28.558777 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:55:28.558794 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 06:55:28.558886 (+ 92us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:55:28.558894 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:55:28.560091 (+ 1197us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:55:28.560094 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:55:28.561155 (+ 1061us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:55:28.561165 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:55:28.562455 (+ 1290us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:55:28.562531 (+ 76us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:55:28.562707 (+ 176us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:55:28.562734 (+ 27us) client_negotiation.cc:769] Sending connection context
0812 06:55:28.562806 (+ 72us) client_negotiation.cc:240] Negotiation successful
0812 06:55:28.563397 (+ 591us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":47,"mutex_wait_us":43}
I20260812 06:55:28.563685 27843 tablet_service.cc:1467] Processing CreateTablet for tablet 94387c2003ed4c5b9a20c9a964d88bec (DEFAULT_TABLE table=test-table [id=69ab4e634335411988d2c62a74a86b43]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 06:55:28.563961 27843 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 94387c2003ed4c5b9a20c9a964d88bec. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 06:55:28.564522 28361 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:28.557217 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.66:34045 (local address 127.0.0.1:51790)
0812 06:55:28.558993 (+ 1776us) negotiation.cc:107] Waiting for socket to connect
0812 06:55:28.559011 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0812 06:55:28.559120 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:55:28.559656 (+ 536us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:55:28.559659 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:55:28.559679 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 06:55:28.559834 (+ 155us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:55:28.559840 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:55:28.561144 (+ 1304us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:55:28.561148 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:55:28.562706 (+ 1558us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:55:28.562716 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:55:28.564105 (+ 1389us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:55:28.564109 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:55:28.564191 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:55:28.564215 (+ 24us) client_negotiation.cc:769] Sending connection context
0812 06:55:28.564303 (+ 88us) client_negotiation.cc:240] Negotiation successful
0812 06:55:28.564391 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1691,"mutex_wait_us":62,"thread_start_us":170,"threads_started":1}
I20260812 06:55:28.564810 28363 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:28.557431 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51790 (local address 127.19.52.66:34045)
0812 06:55:28.557711 (+ 280us) server_negotiation.cc:206] Beginning negotiation
0812 06:55:28.557717 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 06:55:28.559206 (+ 1489us) server_negotiation.cc:407] Connection header received
0812 06:55:28.559266 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:55:28.559269 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:55:28.559319 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:55:28.559392 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 06:55:28.560059 (+ 667us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:28.560997 (+ 938us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:28.562877 (+ 1880us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:28.563944 (+ 1067us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:28.564072 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:55:28.564161 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0812 06:55:28.564340 (+ 179us) server_negotiation.cc:299] Negotiation successful
0812 06:55:28.564624 (+ 284us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":197,"thread_start_us":107,"threads_started":1}
I20260812 06:55:28.565040 28360 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:28.556631 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52166 (local address 127.19.52.67:45705)
0812 06:55:28.558287 (+ 1656us) server_negotiation.cc:206] Beginning negotiation
0812 06:55:28.558291 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 06:55:28.558305 (+ 14us) server_negotiation.cc:407] Connection header received
0812 06:55:28.558352 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:55:28.558355 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:55:28.558397 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:55:28.558747 (+ 350us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 06:55:28.559039 (+ 292us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:28.559920 (+ 881us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:28.561319 (+ 1399us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:28.562192 (+ 873us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:28.562311 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:55:28.562377 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0812 06:55:28.564205 (+ 1828us) server_negotiation.cc:299] Negotiation successful
0812 06:55:28.564335 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1577,"thread_start_us":104,"threads_started":1}
I20260812 06:55:28.565397 28011 tablet_service.cc:1467] Processing CreateTablet for tablet 94387c2003ed4c5b9a20c9a964d88bec (DEFAULT_TABLE table=test-table [id=69ab4e634335411988d2c62a74a86b43]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 06:55:28.565703 28011 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 94387c2003ed4c5b9a20c9a964d88bec. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 06:55:28.565706 28167 tablet_service.cc:1467] Processing CreateTablet for tablet 94387c2003ed4c5b9a20c9a964d88bec (DEFAULT_TABLE table=test-table [id=69ab4e634335411988d2c62a74a86b43]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 06:55:28.565936 28167 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 94387c2003ed4c5b9a20c9a964d88bec. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 06:55:28.568327 28366 tablet_bootstrap.cc:492] T 94387c2003ed4c5b9a20c9a964d88bec P 2e97338a875f4bada5bf4e6301558f78: Bootstrap starting.
I20260812 06:55:28.568897 28367 tablet_bootstrap.cc:492] T 94387c2003ed4c5b9a20c9a964d88bec P fff2f0f0327b47368c203f7f1aab31fd: Bootstrap starting.
I20260812 06:55:28.569424 28366 tablet_bootstrap.cc:654] T 94387c2003ed4c5b9a20c9a964d88bec P 2e97338a875f4bada5bf4e6301558f78: Neither blocks nor log segments found. Creating new log.
I20260812 06:55:28.569870 28367 tablet_bootstrap.cc:654] T 94387c2003ed4c5b9a20c9a964d88bec P fff2f0f0327b47368c203f7f1aab31fd: Neither blocks nor log segments found. Creating new log.
I20260812 06:55:28.570623 28365 tablet_bootstrap.cc:492] T 94387c2003ed4c5b9a20c9a964d88bec P 49e259bbf4094665b4e899332664889c: Bootstrap starting.
I20260812 06:55:28.570920 28366 tablet_bootstrap.cc:492] T 94387c2003ed4c5b9a20c9a964d88bec P 2e97338a875f4bada5bf4e6301558f78: No bootstrap required, opened a new log
I20260812 06:55:28.571019 28366 ts_tablet_manager.cc:1397] T 94387c2003ed4c5b9a20c9a964d88bec P 2e97338a875f4bada5bf4e6301558f78: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260812 06:55:28.571821 28365 tablet_bootstrap.cc:654] T 94387c2003ed4c5b9a20c9a964d88bec P 49e259bbf4094665b4e899332664889c: Neither blocks nor log segments found. Creating new log.
I20260812 06:55:28.571933 28366 raft_consensus.cc:348] T 94387c2003ed4c5b9a20c9a964d88bec P 2e97338a875f4bada5bf4e6301558f78 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fff2f0f0327b47368c203f7f1aab31fd" member_type: VOTER last_known_addr { host: "127.19.52.67" port: 45705 } } peers { permanent_uuid: "49e259bbf4094665b4e899332664889c" member_type: VOTER last_known_addr { host: "127.19.52.65" port: 41417 } } peers { permanent_uuid: "2e97338a875f4bada5bf4e6301558f78" member_type: VOTER last_known_addr { host: "127.19.52.66" port: 34045 } }
I20260812 06:55:28.572083 28366 raft_consensus.cc:374] T 94387c2003ed4c5b9a20c9a964d88bec P 2e97338a875f4bada5bf4e6301558f78 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 06:55:28.572134 28366 raft_consensus.cc:729] T 94387c2003ed4c5b9a20c9a964d88bec P 2e97338a875f4bada5bf4e6301558f78 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2e97338a875f4bada5bf4e6301558f78, State: Initialized, Role: FOLLOWER
I20260812 06:55:28.572332 28366 consensus_queue.cc:260] T 94387c2003ed4c5b9a20c9a964d88bec P 2e97338a875f4bada5bf4e6301558f78 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fff2f0f0327b47368c203f7f1aab31fd" member_type: VOTER last_known_addr { host: "127.19.52.67" port: 45705 } } peers { permanent_uuid: "49e259bbf4094665b4e899332664889c" member_type: VOTER last_known_addr { host: "127.19.52.65" port: 41417 } } peers { permanent_uuid: "2e97338a875f4bada5bf4e6301558f78" member_type: VOTER last_known_addr { host: "127.19.52.66" port: 34045 } }
I20260812 06:55:28.572808 28366 ts_tablet_manager.cc:1428] T 94387c2003ed4c5b9a20c9a964d88bec P 2e97338a875f4bada5bf4e6301558f78: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260812 06:55:28.573242 28367 tablet_bootstrap.cc:492] T 94387c2003ed4c5b9a20c9a964d88bec P fff2f0f0327b47368c203f7f1aab31fd: No bootstrap required, opened a new log
I20260812 06:55:28.573367 28367 ts_tablet_manager.cc:1397] T 94387c2003ed4c5b9a20c9a964d88bec P fff2f0f0327b47368c203f7f1aab31fd: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.001s
I20260812 06:55:28.573930 28367 raft_consensus.cc:348] T 94387c2003ed4c5b9a20c9a964d88bec P fff2f0f0327b47368c203f7f1aab31fd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fff2f0f0327b47368c203f7f1aab31fd" member_type: VOTER last_known_addr { host: "127.19.52.67" port: 45705 } } peers { permanent_uuid: "49e259bbf4094665b4e899332664889c" member_type: VOTER last_known_addr { host: "127.19.52.65" port: 41417 } } peers { permanent_uuid: "2e97338a875f4bada5bf4e6301558f78" member_type: VOTER last_known_addr { host: "127.19.52.66" port: 34045 } }
I20260812 06:55:28.574064 28367 raft_consensus.cc:374] T 94387c2003ed4c5b9a20c9a964d88bec P fff2f0f0327b47368c203f7f1aab31fd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 06:55:28.574105 28367 raft_consensus.cc:729] T 94387c2003ed4c5b9a20c9a964d88bec P fff2f0f0327b47368c203f7f1aab31fd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fff2f0f0327b47368c203f7f1aab31fd, State: Initialized, Role: FOLLOWER
I20260812 06:55:28.574275 28367 consensus_queue.cc:260] T 94387c2003ed4c5b9a20c9a964d88bec P fff2f0f0327b47368c203f7f1aab31fd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fff2f0f0327b47368c203f7f1aab31fd" member_type: VOTER last_known_addr { host: "127.19.52.67" port: 45705 } } peers { permanent_uuid: "49e259bbf4094665b4e899332664889c" member_type: VOTER last_known_addr { host: "127.19.52.65" port: 41417 } } peers { permanent_uuid: "2e97338a875f4bada5bf4e6301558f78" member_type: VOTER last_known_addr { host: "127.19.52.66" port: 34045 } }
I20260812 06:55:28.574677 28367 ts_tablet_manager.cc:1428] T 94387c2003ed4c5b9a20c9a964d88bec P fff2f0f0327b47368c203f7f1aab31fd: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260812 06:55:28.575513 28375 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:28.574990 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.126:34933 (local address 127.19.52.67:54797)
0812 06:55:28.575314 (+ 324us) negotiation.cc:107] Waiting for socket to connect
0812 06:55:28.575390 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.52.126:34933: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":238,"thread_start_us":138,"threads_started":1}
I20260812 06:55:28.576608 28375 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:28.576112 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.126:34933 (local address 127.19.52.67:52659)
0812 06:55:28.576427 (+ 315us) negotiation.cc:107] Waiting for socket to connect
0812 06:55:28.576494 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.52.126:34933: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":237}
I20260812 06:55:28.576730 28087 heartbeater.cc:503] Master 127.19.52.125:32823 was elected leader, sending a full tablet report...
I20260812 06:55:28.577024 28245 heartbeater.cc:503] Master 127.19.52.125:32823 was elected leader, sending a full tablet report...
I20260812 06:55:28.577219 28373 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:28.573388 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.126:34933 (local address 127.19.52.66:43619)
0812 06:55:28.577007 (+ 3619us) negotiation.cc:107] Waiting for socket to connect
0812 06:55:28.577079 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.52.126:34933: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":3541,"thread_start_us":115,"threads_started":1}
I20260812 06:55:28.577574 28365 tablet_bootstrap.cc:492] T 94387c2003ed4c5b9a20c9a964d88bec P 49e259bbf4094665b4e899332664889c: No bootstrap required, opened a new log
I20260812 06:55:28.577726 28365 ts_tablet_manager.cc:1397] T 94387c2003ed4c5b9a20c9a964d88bec P 49e259bbf4094665b4e899332664889c: Time spent bootstrapping tablet: real 0.007s user 0.003s sys 0.000s
I20260812 06:55:28.578256 28365 raft_consensus.cc:348] T 94387c2003ed4c5b9a20c9a964d88bec P 49e259bbf4094665b4e899332664889c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fff2f0f0327b47368c203f7f1aab31fd" member_type: VOTER last_known_addr { host: "127.19.52.67" port: 45705 } } peers { permanent_uuid: "49e259bbf4094665b4e899332664889c" member_type: VOTER last_known_addr { host: "127.19.52.65" port: 41417 } } peers { permanent_uuid: "2e97338a875f4bada5bf4e6301558f78" member_type: VOTER last_known_addr { host: "127.19.52.66" port: 34045 } }
I20260812 06:55:28.578388 28365 raft_consensus.cc:374] T 94387c2003ed4c5b9a20c9a964d88bec P 49e259bbf4094665b4e899332664889c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 06:55:28.578521 28365 raft_consensus.cc:729] T 94387c2003ed4c5b9a20c9a964d88bec P 49e259bbf4094665b4e899332664889c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 49e259bbf4094665b4e899332664889c, State: Initialized, Role: FOLLOWER
I20260812 06:55:28.579016 28373 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:28.577688 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.126:34933 (local address 127.19.52.66:40063)
0812 06:55:28.578855 (+ 1167us) negotiation.cc:107] Waiting for socket to connect
0812 06:55:28.578930 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.52.126:34933: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1062}
I20260812 06:55:28.578967 28365 consensus_queue.cc:260] T 94387c2003ed4c5b9a20c9a964d88bec P 49e259bbf4094665b4e899332664889c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fff2f0f0327b47368c203f7f1aab31fd" member_type: VOTER last_known_addr { host: "127.19.52.67" port: 45705 } } peers { permanent_uuid: "49e259bbf4094665b4e899332664889c" member_type: VOTER last_known_addr { host: "127.19.52.65" port: 41417 } } peers { permanent_uuid: "2e97338a875f4bada5bf4e6301558f78" member_type: VOTER last_known_addr { host: "127.19.52.66" port: 34045 } }
I20260812 06:55:28.579389 28365 ts_tablet_manager.cc:1428] T 94387c2003ed4c5b9a20c9a964d88bec P 49e259bbf4094665b4e899332664889c: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260812 06:55:28.580252 28377 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:28.579678 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.126:34933 (local address 127.19.52.65:33591)
0812 06:55:28.580084 (+ 406us) negotiation.cc:107] Waiting for socket to connect
0812 06:55:28.580148 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.52.126:34933: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":334,"thread_start_us":136,"threads_started":1}
I20260812 06:55:28.580900 27925 heartbeater.cc:503] Master 127.19.52.125:32823 was elected leader, sending a full tablet report...
I20260812 06:55:28.581089 28377 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:28.580773 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.126:34933 (local address 127.19.52.65:36453)
0812 06:55:28.580903 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0812 06:55:28.580964 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.52.126:34933: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":62}
I20260812 06:55:28.620797 28376 raft_consensus.cc:482] T 94387c2003ed4c5b9a20c9a964d88bec P 49e259bbf4094665b4e899332664889c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 06:55:28.620930 28376 raft_consensus.cc:504] T 94387c2003ed4c5b9a20c9a964d88bec P 49e259bbf4094665b4e899332664889c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fff2f0f0327b47368c203f7f1aab31fd" member_type: VOTER last_known_addr { host: "127.19.52.67" port: 45705 } } peers { permanent_uuid: "49e259bbf4094665b4e899332664889c" member_type: VOTER last_known_addr { host: "127.19.52.65" port: 41417 } } peers { permanent_uuid: "2e97338a875f4bada5bf4e6301558f78" member_type: VOTER last_known_addr { host: "127.19.52.66" port: 34045 } }
I20260812 06:55:28.621398 28376 leader_election.cc:288] T 94387c2003ed4c5b9a20c9a964d88bec P 49e259bbf4094665b4e899332664889c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fff2f0f0327b47368c203f7f1aab31fd (127.19.52.67:45705), 2e97338a875f4bada5bf4e6301558f78 (127.19.52.66:34045)
I20260812 06:55:28.621891 28030 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "94387c2003ed4c5b9a20c9a964d88bec" candidate_uuid: "49e259bbf4094665b4e899332664889c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2e97338a875f4bada5bf4e6301558f78" is_pre_election: true
I20260812 06:55:28.622027 28030 raft_consensus.cc:2454] T 94387c2003ed4c5b9a20c9a964d88bec P 2e97338a875f4bada5bf4e6301558f78 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49e259bbf4094665b4e899332664889c in term 0.
I20260812 06:55:28.622006 28191 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "94387c2003ed4c5b9a20c9a964d88bec" candidate_uuid: "49e259bbf4094665b4e899332664889c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fff2f0f0327b47368c203f7f1aab31fd" is_pre_election: true
I20260812 06:55:28.622187 28191 raft_consensus.cc:2454] T 94387c2003ed4c5b9a20c9a964d88bec P fff2f0f0327b47368c203f7f1aab31fd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49e259bbf4094665b4e899332664889c in term 0.
I20260812 06:55:28.622385 27791 leader_election.cc:302] T 94387c2003ed4c5b9a20c9a964d88bec P 49e259bbf4094665b4e899332664889c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2e97338a875f4bada5bf4e6301558f78, 49e259bbf4094665b4e899332664889c; no voters:
I20260812 06:55:28.622599 28376 raft_consensus.cc:2781] T 94387c2003ed4c5b9a20c9a964d88bec P 49e259bbf4094665b4e899332664889c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 06:55:28.622664 28376 raft_consensus.cc:482] T 94387c2003ed4c5b9a20c9a964d88bec P 49e259bbf4094665b4e899332664889c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 06:55:28.622705 28376 raft_consensus.cc:3037] T 94387c2003ed4c5b9a20c9a964d88bec P 49e259bbf4094665b4e899332664889c [term 0 FOLLOWER]: Advancing to term 1
I20260812 06:55:28.623696 28376 raft_consensus.cc:504] T 94387c2003ed4c5b9a20c9a964d88bec P 49e259bbf4094665b4e899332664889c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fff2f0f0327b47368c203f7f1aab31fd" member_type: VOTER last_known_addr { host: "127.19.52.67" port: 45705 } } peers { permanent_uuid: "49e259bbf4094665b4e899332664889c" member_type: VOTER last_known_addr { host: "127.19.52.65" port: 41417 } } peers { permanent_uuid: "2e97338a875f4bada5bf4e6301558f78" member_type: VOTER last_known_addr { host: "127.19.52.66" port: 34045 } }
I20260812 06:55:28.624023 28376 leader_election.cc:288] T 94387c2003ed4c5b9a20c9a964d88bec P 49e259bbf4094665b4e899332664889c [CANDIDATE]: Term 1 election: Requested vote from peers fff2f0f0327b47368c203f7f1aab31fd (127.19.52.67:45705), 2e97338a875f4bada5bf4e6301558f78 (127.19.52.66:34045)
I20260812 06:55:28.624377 28030 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "94387c2003ed4c5b9a20c9a964d88bec" candidate_uuid: "49e259bbf4094665b4e899332664889c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2e97338a875f4bada5bf4e6301558f78"
I20260812 06:55:28.624364 28191 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "94387c2003ed4c5b9a20c9a964d88bec" candidate_uuid: "49e259bbf4094665b4e899332664889c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fff2f0f0327b47368c203f7f1aab31fd"
I20260812 06:55:28.624495 28030 raft_consensus.cc:3037] T 94387c2003ed4c5b9a20c9a964d88bec P 2e97338a875f4bada5bf4e6301558f78 [term 0 FOLLOWER]: Advancing to term 1
I20260812 06:55:28.624529 28191 raft_consensus.cc:3037] T 94387c2003ed4c5b9a20c9a964d88bec P fff2f0f0327b47368c203f7f1aab31fd [term 0 FOLLOWER]: Advancing to term 1
I20260812 06:55:28.625479 28030 raft_consensus.cc:2454] T 94387c2003ed4c5b9a20c9a964d88bec P 2e97338a875f4bada5bf4e6301558f78 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49e259bbf4094665b4e899332664889c in term 1.
I20260812 06:55:28.625571 28191 raft_consensus.cc:2454] T 94387c2003ed4c5b9a20c9a964d88bec P fff2f0f0327b47368c203f7f1aab31fd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49e259bbf4094665b4e899332664889c in term 1.
I20260812 06:55:28.625769 27791 leader_election.cc:302] T 94387c2003ed4c5b9a20c9a964d88bec P 49e259bbf4094665b4e899332664889c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2e97338a875f4bada5bf4e6301558f78, 49e259bbf4094665b4e899332664889c; no voters:
I20260812 06:55:28.625944 28376 raft_consensus.cc:2781] T 94387c2003ed4c5b9a20c9a964d88bec P 49e259bbf4094665b4e899332664889c [term 1 FOLLOWER]: Leader election won for term 1
I20260812 06:55:28.626164 28376 raft_consensus.cc:686] T 94387c2003ed4c5b9a20c9a964d88bec P 49e259bbf4094665b4e899332664889c [term 1 LEADER]: Becoming Leader. State: Replica: 49e259bbf4094665b4e899332664889c, State: Running, Role: LEADER
I20260812 06:55:28.626319 28376 consensus_queue.cc:237] T 94387c2003ed4c5b9a20c9a964d88bec P 49e259bbf4094665b4e899332664889c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fff2f0f0327b47368c203f7f1aab31fd" member_type: VOTER last_known_addr { host: "127.19.52.67" port: 45705 } } peers { permanent_uuid: "49e259bbf4094665b4e899332664889c" member_type: VOTER last_known_addr { host: "127.19.52.65" port: 41417 } } peers { permanent_uuid: "2e97338a875f4bada5bf4e6301558f78" member_type: VOTER last_known_addr { host: "127.19.52.66" port: 34045 } }
I20260812 06:55:28.626940 28377 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:28.626686 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.126:34933 (local address 127.19.52.65:58107)
0812 06:55:28.626811 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0812 06:55:28.626873 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.52.126:34933: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
I20260812 06:55:28.628216 27502 catalog_manager.cc:5495] T 94387c2003ed4c5b9a20c9a964d88bec P 49e259bbf4094665b4e899332664889c reported cstate change: term changed from 0 to 1, leader changed from <none> to 49e259bbf4094665b4e899332664889c (127.19.52.65). New cstate: current_term: 1 leader_uuid: "49e259bbf4094665b4e899332664889c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fff2f0f0327b47368c203f7f1aab31fd" member_type: VOTER last_known_addr { host: "127.19.52.67" port: 45705 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "49e259bbf4094665b4e899332664889c" member_type: VOTER last_known_addr { host: "127.19.52.65" port: 41417 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2e97338a875f4bada5bf4e6301558f78" member_type: VOTER last_known_addr { host: "127.19.52.66" port: 34045 } health_report { overall_health: UNKNOWN } } }
I20260812 06:55:28.643262 28362 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:28.639812 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54914 (local address 127.19.52.65:41417)
0812 06:55:28.639954 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0812 06:55:28.639958 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 06:55:28.639974 (+ 16us) server_negotiation.cc:407] Connection header received
0812 06:55:28.640086 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:55:28.640089 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:55:28.640146 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:55:28.640213 (+ 67us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 06:55:28.640504 (+ 291us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:28.641116 (+ 612us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:28.642106 (+ 990us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:28.642325 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:28.642430 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:55:28.642719 (+ 289us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 06:55:28.642799 (+ 80us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 06:55:28.642869 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0812 06:55:28.642990 (+ 121us) server_negotiation.cc:299] Negotiation successful
0812 06:55:28.643041 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260812 06:55:28.645689 28191 raft_consensus.cc:1261] T 94387c2003ed4c5b9a20c9a964d88bec P fff2f0f0327b47368c203f7f1aab31fd [term 1 FOLLOWER]: Refusing update from remote peer 49e259bbf4094665b4e899332664889c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 06:55:28.645706 28030 raft_consensus.cc:1261] T 94387c2003ed4c5b9a20c9a964d88bec P 2e97338a875f4bada5bf4e6301558f78 [term 1 FOLLOWER]: Refusing update from remote peer 49e259bbf4094665b4e899332664889c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 06:55:28.646196 28376 consensus_queue.cc:1035] T 94387c2003ed4c5b9a20c9a964d88bec P 49e259bbf4094665b4e899332664889c [LEADER]: Connected to new peer: Peer: permanent_uuid: "fff2f0f0327b47368c203f7f1aab31fd" member_type: VOTER last_known_addr { host: "127.19.52.67" port: 45705 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 06:55:28.646395 28378 consensus_queue.cc:1035] T 94387c2003ed4c5b9a20c9a964d88bec P 49e259bbf4094665b4e899332664889c [LEADER]: Connected to new peer: Peer: permanent_uuid: "2e97338a875f4bada5bf4e6301558f78" member_type: VOTER last_known_addr { host: "127.19.52.66" port: 34045 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 06:55:28.648627 28375 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:28.648000 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.126:34933 (local address 127.19.52.67:53511)
0812 06:55:28.648438 (+ 438us) negotiation.cc:107] Waiting for socket to connect
0812 06:55:28.648510 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.52.126:34933: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":360}
I20260812 06:55:28.648983 28373 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:28.648275 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.126:34933 (local address 127.19.52.66:55083)
0812 06:55:28.648850 (+ 575us) negotiation.cc:107] Waiting for socket to connect
0812 06:55:28.648909 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.52.126:34933: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":485}
I20260812 06:55:28.651058 28377 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:28.650805 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.126:34933 (local address 127.19.52.65:46357)
0812 06:55:28.650922 (+ 117us) negotiation.cc:107] Waiting for socket to connect
0812 06:55:28.650982 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.52.126:34933: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
I20260812 06:55:28.651991 28375 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:28.651514 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.126:34933 (local address 127.19.52.67:48673)
0812 06:55:28.651824 (+ 310us) negotiation.cc:107] Waiting for socket to connect
0812 06:55:28.651882 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.52.126:34933: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":239}
I20260812 06:55:28.653929 28377 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:28.653471 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.126:34933 (local address 127.19.52.65:38945)
0812 06:55:28.653787 (+ 316us) negotiation.cc:107] Waiting for socket to connect
0812 06:55:28.653847 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.52.126:34933: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":242}
I20260812 06:55:28.655431 28373 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:28.655024 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.126:34933 (local address 127.19.52.66:58519)
0812 06:55:28.655318 (+ 294us) negotiation.cc:107] Waiting for socket to connect
0812 06:55:28.655366 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.52.126:34933: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":224}
I20260812 06:55:28.658536 27502 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55298:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 06:55:28.658696 27502 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55298:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 06:55:28.663445 27502 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 2b5a510eb7be4cfab2f7914ebabaa929: Sending DeleteTablet for 3 replicas of tablet 94387c2003ed4c5b9a20c9a964d88bec
I20260812 06:55:28.664295 27843 tablet_service.cc:1514] Processing DeleteTablet for tablet 94387c2003ed4c5b9a20c9a964d88bec with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 06:55:28 UTC) from {username='oryx', principal='oryx/127.19.52.125@KRBTEST.COM'} at 127.0.0.1:54904
I20260812 06:55:28.664322 28167 tablet_service.cc:1514] Processing DeleteTablet for tablet 94387c2003ed4c5b9a20c9a964d88bec with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 06:55:28 UTC) from {username='oryx', principal='oryx/127.19.52.125@KRBTEST.COM'} at 127.0.0.1:52166
I20260812 06:55:28.664482 28011 tablet_service.cc:1514] Processing DeleteTablet for tablet 94387c2003ed4c5b9a20c9a964d88bec with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 06:55:28 UTC) from {username='oryx', principal='oryx/127.19.52.125@KRBTEST.COM'} at 127.0.0.1:51790
I20260812 06:55:28.664816 28393 tablet_replica.cc:330] T 94387c2003ed4c5b9a20c9a964d88bec P 49e259bbf4094665b4e899332664889c: stopping tablet replica
I20260812 06:55:28.664876 28395 tablet_replica.cc:330] T 94387c2003ed4c5b9a20c9a964d88bec P 2e97338a875f4bada5bf4e6301558f78: stopping tablet replica
I20260812 06:55:28.664974 28393 raft_consensus.cc:2229] T 94387c2003ed4c5b9a20c9a964d88bec P 49e259bbf4094665b4e899332664889c [term 1 LEADER]: Raft consensus shutting down.
I20260812 06:55:28.664991 28395 raft_consensus.cc:2229] T 94387c2003ed4c5b9a20c9a964d88bec P 2e97338a875f4bada5bf4e6301558f78 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 06:55:28.665120 28395 raft_consensus.cc:2258] T 94387c2003ed4c5b9a20c9a964d88bec P 2e97338a875f4bada5bf4e6301558f78 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 06:55:28.665194 28393 raft_consensus.cc:2258] T 94387c2003ed4c5b9a20c9a964d88bec P 49e259bbf4094665b4e899332664889c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 06:55:28.665452 28394 tablet_replica.cc:330] T 94387c2003ed4c5b9a20c9a964d88bec P fff2f0f0327b47368c203f7f1aab31fd: stopping tablet replica
I20260812 06:55:28.665582 19665 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu with pid 27669
I20260812 06:55:28.665596 28394 raft_consensus.cc:2229] T 94387c2003ed4c5b9a20c9a964d88bec P fff2f0f0327b47368c203f7f1aab31fd [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 06:55:28.665743 28394 raft_consensus.cc:2258] T 94387c2003ed4c5b9a20c9a964d88bec P fff2f0f0327b47368c203f7f1aab31fd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 06:55:28.668635 28394 ts_tablet_manager.cc:1905] T 94387c2003ed4c5b9a20c9a964d88bec P fff2f0f0327b47368c203f7f1aab31fd: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 06:55:28.670974 28395 ts_tablet_manager.cc:1905] T 94387c2003ed4c5b9a20c9a964d88bec P 2e97338a875f4bada5bf4e6301558f78: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 06:55:28.672747 28394 ts_tablet_manager.cc:1918] T 94387c2003ed4c5b9a20c9a964d88bec P fff2f0f0327b47368c203f7f1aab31fd: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 06:55:28.672860 28394 log.cc:1196] T 94387c2003ed4c5b9a20c9a964d88bec P fff2f0f0327b47368c203f7f1aab31fd: Deleting WAL directory at /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517687384214-19665-0/minicluster-data/ts-2/wal/wals/94387c2003ed4c5b9a20c9a964d88bec
I20260812 06:55:28.673166 28394 ts_tablet_manager.cc:1939] T 94387c2003ed4c5b9a20c9a964d88bec P fff2f0f0327b47368c203f7f1aab31fd: Deleting consensus metadata
I20260812 06:55:28.674239 28395 ts_tablet_manager.cc:1918] T 94387c2003ed4c5b9a20c9a964d88bec P 2e97338a875f4bada5bf4e6301558f78: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 06:55:28.674345 28395 log.cc:1196] T 94387c2003ed4c5b9a20c9a964d88bec P 2e97338a875f4bada5bf4e6301558f78: Deleting WAL directory at /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517687384214-19665-0/minicluster-data/ts-1/wal/wals/94387c2003ed4c5b9a20c9a964d88bec
I20260812 06:55:28.674321 27485 catalog_manager.cc:4841] TS fff2f0f0327b47368c203f7f1aab31fd (127.19.52.67:45705): tablet 94387c2003ed4c5b9a20c9a964d88bec (table test-table [id=69ab4e634335411988d2c62a74a86b43]) successfully deleted
I20260812 06:55:28.674680 28395 ts_tablet_manager.cc:1939] T 94387c2003ed4c5b9a20c9a964d88bec P 2e97338a875f4bada5bf4e6301558f78: Deleting consensus metadata
I20260812 06:55:28.675472 27484 catalog_manager.cc:4841] TS 2e97338a875f4bada5bf4e6301558f78 (127.19.52.66:34045): tablet 94387c2003ed4c5b9a20c9a964d88bec (table test-table [id=69ab4e634335411988d2c62a74a86b43]) successfully deleted
I20260812 06:55:28.681717 19665 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu with pid 27932
I20260812 06:55:28.683281 28361 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:28.682032 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.65:41417 (local address 127.0.0.1:54924)
0812 06:55:28.683090 (+ 1058us) negotiation.cc:107] Waiting for socket to connect
0812 06:55:28.683165 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.52.65:41417: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":937}
W20260812 06:55:28.683673 27483 catalog_manager.cc:4584] TS 49e259bbf4094665b4e899332664889c (127.19.52.65:41417): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 94387c2003ed4c5b9a20c9a964d88bec: Network error: Client connection negotiation failed: client connection to 127.19.52.65:41417: connect: Connection refused (error 111)
I20260812 06:55:28.690517 19665 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu with pid 28093
I20260812 06:55:28.700634 19665 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu with pid 27340
I20260812 06:55:28.710012 19665 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu with pid 27456
2026-08-12T06:55:28Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipalMultipleMaster (7728 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786517687384214-19665-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 06:55:28 dist-test-slave-65mx krb5kdc[28399](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 06:55:28 dist-test-slave-65mx krb5kdc[28399](info): set up 2 sockets
Aug 12 06:55:28 dist-test-slave-65mx krb5kdc[28399](info): commencing operation
krb5kdc: starting...
W20260812 06:55:29.878456 19665 mini_kdc.cc:121] Time spent starting KDC: real 1.140s user 0.004s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786517687384214-19665-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786517687384214-19665-0/krb5kdc/test-user.keytab.
Aug 12 06:55:29 dist-test-slave-65mx krb5kdc[28399](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517729, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T06:55:29Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T06:55:29Z Disabled control of system clock
WARNING: no policy specified for oryx/127.19.52.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.19.52.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.19.52.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786517687384214-19665-0/krb5kdc/oryx_127.19.52.126.keytab.
Entry for principal oryx/127.19.52.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786517687384214-19665-0/krb5kdc/oryx_127.19.52.126.keytab.
I20260812 06:55:30.020673 19665 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu
/tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786517687384214-19665-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786517687384214-19665-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786517687384214-19665-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786517687384214-19665-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.52.126:41321
--webserver_interface=127.19.52.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786517687384214-19665-0/krb5kdc/oryx_127.19.52.126.keytab
--principal=oryx/127.19.52.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.52.84:35457
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.19.52.126:41321
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786517687384214-19665-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786517687384214-19665-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786517687384214-19665-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 06:55:30.136927 28412 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:55:30.137256 28412 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:55:30.137336 28412 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 06:55:30.141708 28412 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 06:55:30.141841 28412 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260812 06:55:30.141880 28412 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 06:55:30.141924 28412 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 06:55:30.141959 28412 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 06:55:30.142009 28412 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 06:55:30.147691 28412 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.52.84:35457
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786517687384214-19665-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786517687384214-19665-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.19.52.126:41321
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786517687384214-19665-0/krb5kdc/oryx_127.19.52.126.keytab
--principal=oryx/127.19.52.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.52.126:41321
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786517687384214-19665-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.19.52.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28412
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786517687384214-19665-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 06:15:54 UTC on ccycloud.quasar-rnfzce.root.comops.site
build id 4581989
I20260812 06:55:30.148959 28412 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:55:30.150048 28412 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 06:55:30.160228 28418 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:55:30.160315 28420 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:55:30.160310 28412 server_base.cc:1034] running on GCE node
W20260812 06:55:30.160468 28417 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:55:30.161302 28412 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:55:30.162595 28412 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:55:30.163779 28412 hybrid_clock.cc:648] HybridClock initialized: now 1786517730163745 us; error 53 us; skew 500 ppm
Aug 12 06:55:30 dist-test-slave-65mx krb5kdc[28399](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517730, etypes {rep=17 tkt=17 ses=17}, oryx/127.19.52.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 06:55:30.170732 28412 init.cc:377] Logged in from keytab as oryx/127.19.52.126@KRBTEST.COM (short username oryx)
I20260812 06:55:30.171754 28412 webserver.cc:460] Webserver started at http://127.19.52.126:33923/ using document root <none> and password file <none>
I20260812 06:55:30.172328 28412 fs_manager.cc:359] Metadata directory not provided
I20260812 06:55:30.172389 28412 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:55:30.172636 28412 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 06:55:30.175093 28412 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786517687384214-19665-0/minicluster-data/master-0/data/instance:
uuid: "3bce2c30f41c4196af2accc6509b44ff"
format_stamp: "Formatted at 2026-08-12 06:55:30 on dist-test-slave-65mx"
server_key: "823e5958bacd68f17622554284eaca7d"
server_key_iv: "36c96a4e4030ed31f2404949c2a0e73d"
server_key_version: "encryptionkey@0"
I20260812 06:55:30.175730 28412 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786517687384214-19665-0/minicluster-data/master-0/wal/instance:
uuid: "3bce2c30f41c4196af2accc6509b44ff"
format_stamp: "Formatted at 2026-08-12 06:55:30 on dist-test-slave-65mx"
server_key: "823e5958bacd68f17622554284eaca7d"
server_key_iv: "36c96a4e4030ed31f2404949c2a0e73d"
server_key_version: "encryptionkey@0"
I20260812 06:55:30.180423 28412 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.002s
I20260812 06:55:30.183421 28429 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 06:55:30.185176 28412 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 06:55:30.185314 28412 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786517687384214-19665-0/minicluster-data/master-0/data,/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786517687384214-19665-0/minicluster-data/master-0/wal
uuid: "3bce2c30f41c4196af2accc6509b44ff"
format_stamp: "Formatted at 2026-08-12 06:55:30 on dist-test-slave-65mx"
server_key: "823e5958bacd68f17622554284eaca7d"
server_key_iv: "36c96a4e4030ed31f2404949c2a0e73d"
server_key_version: "encryptionkey@0"
I20260812 06:55:30.185427 28412 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786517687384214-19665-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786517687384214-19665-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786517687384214-19665-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:55:30.227144 28412 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 06:55:30.230370 28412 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 06:55:30.230538 28412 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:55:30.239223 28412 rpc_server.cc:307] RPC server started. Bound to: 127.19.52.126:41321
I20260812 06:55:30.239317 28500 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.52.126:41321 every 8 connection(s)
I20260812 06:55:30.240242 28412 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786517687384214-19665-0/minicluster-data/master-0/data/info.pb
I20260812 06:55:30.243278 28501 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 06:55:30.249127 28501 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3bce2c30f41c4196af2accc6509b44ff: Bootstrap starting.
I20260812 06:55:30.249166 19665 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu as pid 28412
I20260812 06:55:30.249321 19665 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786517687384214-19665-0/minicluster-data/master-0/wal/instance
I20260812 06:55:30.249620 19665 external_mini_cluster.cc:1428] Setting key a814737290e742db5c087f68aec0e057
I20260812 06:55:30.251922 28501 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3bce2c30f41c4196af2accc6509b44ff: Neither blocks nor log segments found. Creating new log.
I20260812 06:55:30.252810 28501 log.cc:824] T 00000000000000000000000000000000 P 3bce2c30f41c4196af2accc6509b44ff: Log is configured to *not* fsync() on all Append() calls
I20260812 06:55:30.255208 28501 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3bce2c30f41c4196af2accc6509b44ff: No bootstrap required, opened a new log
Aug 12 06:55:30 dist-test-slave-65mx krb5kdc[28399](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517729, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.19.52.126@KRBTEST.COM
I20260812 06:55:30.260105 28501 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3bce2c30f41c4196af2accc6509b44ff [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3bce2c30f41c4196af2accc6509b44ff" member_type: VOTER last_known_addr { host: "127.19.52.126" port: 41321 } }
I20260812 06:55:30.260408 28501 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3bce2c30f41c4196af2accc6509b44ff [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 06:55:30.260509 28501 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3bce2c30f41c4196af2accc6509b44ff [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3bce2c30f41c4196af2accc6509b44ff, State: Initialized, Role: FOLLOWER
I20260812 06:55:30.261088 28501 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3bce2c30f41c4196af2accc6509b44ff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3bce2c30f41c4196af2accc6509b44ff" member_type: VOTER last_known_addr { host: "127.19.52.126" port: 41321 } }
I20260812 06:55:30.261282 28501 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3bce2c30f41c4196af2accc6509b44ff [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 06:55:30.261380 28501 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3bce2c30f41c4196af2accc6509b44ff [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 06:55:30.261497 28501 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3bce2c30f41c4196af2accc6509b44ff [term 0 FOLLOWER]: Advancing to term 1
I20260812 06:55:30.262835 28501 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3bce2c30f41c4196af2accc6509b44ff [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3bce2c30f41c4196af2accc6509b44ff" member_type: VOTER last_known_addr { host: "127.19.52.126" port: 41321 } }
I20260812 06:55:30.263259 28501 leader_election.cc:302] T 00000000000000000000000000000000 P 3bce2c30f41c4196af2accc6509b44ff [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3bce2c30f41c4196af2accc6509b44ff; no voters:
I20260812 06:55:30.263690 28501 leader_election.cc:288] T 00000000000000000000000000000000 P 3bce2c30f41c4196af2accc6509b44ff [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 06:55:30.263821 28506 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3bce2c30f41c4196af2accc6509b44ff [term 1 FOLLOWER]: Leader election won for term 1
I20260812 06:55:30.264091 28506 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3bce2c30f41c4196af2accc6509b44ff [term 1 LEADER]: Becoming Leader. State: Replica: 3bce2c30f41c4196af2accc6509b44ff, State: Running, Role: LEADER
I20260812 06:55:30.264425 28506 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3bce2c30f41c4196af2accc6509b44ff [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3bce2c30f41c4196af2accc6509b44ff" member_type: VOTER last_known_addr { host: "127.19.52.126" port: 41321 } }
I20260812 06:55:30.265270 28501 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3bce2c30f41c4196af2accc6509b44ff [sys.catalog]: configured and running, proceeding with master startup.
I20260812 06:55:30.266223 28507 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3bce2c30f41c4196af2accc6509b44ff [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3bce2c30f41c4196af2accc6509b44ff" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3bce2c30f41c4196af2accc6509b44ff" member_type: VOTER last_known_addr { host: "127.19.52.126" port: 41321 } } }
I20260812 06:55:30.266359 28507 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3bce2c30f41c4196af2accc6509b44ff [sys.catalog]: This master's current role is: LEADER
I20260812 06:55:30.266659 28508 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3bce2c30f41c4196af2accc6509b44ff [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3bce2c30f41c4196af2accc6509b44ff. Latest consensus state: current_term: 1 leader_uuid: "3bce2c30f41c4196af2accc6509b44ff" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3bce2c30f41c4196af2accc6509b44ff" member_type: VOTER last_known_addr { host: "127.19.52.126" port: 41321 } } }
I20260812 06:55:30.266736 28508 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3bce2c30f41c4196af2accc6509b44ff [sys.catalog]: This master's current role is: LEADER
I20260812 06:55:30.267006 28515 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 06:55:30.267431 28504 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:30.251059 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42076 (local address 127.19.52.126:41321)
0812 06:55:30.251634 (+ 575us) server_negotiation.cc:206] Beginning negotiation
0812 06:55:30.251656 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0812 06:55:30.251694 (+ 38us) server_negotiation.cc:407] Connection header received
0812 06:55:30.252431 (+ 737us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:55:30.252449 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:55:30.252879 (+ 430us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:55:30.253269 (+ 390us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:55:30.254300 (+ 1031us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:30.255581 (+ 1281us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:30.256808 (+ 1227us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:30.257316 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:30.257452 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:55:30.260763 (+ 3311us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:55:30.260769 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:55:30.260787 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:55:30.260825 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:55:30.263304 (+ 2479us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:55:30.263949 (+ 645us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:55:30.263958 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:55:30.263964 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:55:30.264122 (+ 158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:55:30.264593 (+ 471us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:55:30.264597 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:55:30.264599 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:55:30.265166 (+ 567us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:55:30.265375 (+ 209us) server_negotiation.cc:1035] Waiting for connection context
0812 06:55:30.265942 (+ 567us) server_negotiation.cc:299] Negotiation successful
0812 06:55:30.266325 (+ 383us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":314,"thread_start_us":130,"threads_started":1}
I20260812 06:55:30.272329 28515 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 06:55:30.281482 28515 catalog_manager.cc:1347] Generated new cluster ID: 44089b086db64aeaa2883c75c73775ad
I20260812 06:55:30.281677 28515 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 06:55:30.288959 28515 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 06:55:30.290041 28515 catalog_manager.cc:1504] Loading token signing keys...
I20260812 06:55:30.296038 28515 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3bce2c30f41c4196af2accc6509b44ff: Generated new TSK 0
I20260812 06:55:30.296864 28515 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.19.52.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.19.52.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.19.52.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786517687384214-19665-0/krb5kdc/oryx_127.19.52.65.keytab.
Entry for principal oryx/127.19.52.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786517687384214-19665-0/krb5kdc/oryx_127.19.52.65.keytab.
I20260812 06:55:30.340318 19665 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu
/tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786517687384214-19665-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786517687384214-19665-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786517687384214-19665-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786517687384214-19665-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.52.65:0
--local_ip_for_outbound_sockets=127.19.52.65
--webserver_interface=127.19.52.65
--webserver_port=0
--tserver_master_addrs=127.19.52.126:41321
--keytab_file=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786517687384214-19665-0/krb5kdc/oryx_127.19.52.65.keytab
--principal=oryx/127.19.52.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.52.84:35457
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786517687384214-19665-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786517687384214-19665-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786517687384214-19665-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 06:55:30.480715 28531 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:55:30.481040 28531 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:55:30.481089 28531 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 06:55:30.485412 28531 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 06:55:30.485489 28531 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 06:55:30.485530 28531 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260812 06:55:30.485611 28531 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.52.65
I20260812 06:55:30.491988 28531 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.52.84:35457
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786517687384214-19665-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786517687384214-19665-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786517687384214-19665-0/krb5kdc/oryx_127.19.52.65.keytab
--principal=oryx/127.19.52.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.52.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786517687384214-19665-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.19.52.65
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.19.52.126:41321
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28531
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.52.65
--log_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786517687384214-19665-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 06:15:54 UTC on ccycloud.quasar-rnfzce.root.comops.site
build id 4581989
I20260812 06:55:30.493243 28531 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:55:30.494314 28531 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260812 06:55:30.504743 28531 server_base.cc:1034] running on GCE node
W20260812 06:55:30.504727 28537 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:55:30.504709 28536 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:55:30.504935 28540 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:55:30.505369 28531 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:55:30.506032 28531 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:55:30.507208 28531 hybrid_clock.cc:648] HybridClock initialized: now 1786517730507176 us; error 46 us; skew 500 ppm
Aug 12 06:55:30 dist-test-slave-65mx krb5kdc[28399](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517730, etypes {rep=17 tkt=17 ses=17}, oryx/127.19.52.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 06:55:30.515175 28531 init.cc:377] Logged in from keytab as oryx/127.19.52.65@KRBTEST.COM (short username oryx)
I20260812 06:55:30.516224 28531 webserver.cc:460] Webserver started at http://127.19.52.65:37183/ using document root <none> and password file <none>
I20260812 06:55:30.516927 28531 fs_manager.cc:359] Metadata directory not provided
I20260812 06:55:30.517032 28531 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:55:30.517279 28531 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 06:55:30.519448 28531 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786517687384214-19665-0/minicluster-data/ts-0/data/instance:
uuid: "2cebfda268094256bb864f37e363b58c"
format_stamp: "Formatted at 2026-08-12 06:55:30 on dist-test-slave-65mx"
server_key: "6f7d023751fbb76d8f35ea4ef9aae27b"
server_key_iv: "7f7bdf254f1c0d2b8f1a6637dbe05ddf"
server_key_version: "encryptionkey@0"
I20260812 06:55:30.520095 28531 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786517687384214-19665-0/minicluster-data/ts-0/wal/instance:
uuid: "2cebfda268094256bb864f37e363b58c"
format_stamp: "Formatted at 2026-08-12 06:55:30 on dist-test-slave-65mx"
server_key: "6f7d023751fbb76d8f35ea4ef9aae27b"
server_key_iv: "7f7bdf254f1c0d2b8f1a6637dbe05ddf"
server_key_version: "encryptionkey@0"
I20260812 06:55:30.524253 28531 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.003s
I20260812 06:55:30.526888 28547 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 06:55:30.528210 28531 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260812 06:55:30.528369 28531 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786517687384214-19665-0/minicluster-data/ts-0/data,/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786517687384214-19665-0/minicluster-data/ts-0/wal
uuid: "2cebfda268094256bb864f37e363b58c"
format_stamp: "Formatted at 2026-08-12 06:55:30 on dist-test-slave-65mx"
server_key: "6f7d023751fbb76d8f35ea4ef9aae27b"
server_key_iv: "7f7bdf254f1c0d2b8f1a6637dbe05ddf"
server_key_version: "encryptionkey@0"
I20260812 06:55:30.528498 28531 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786517687384214-19665-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786517687384214-19665-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786517687384214-19665-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:55:30.538496 28531 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 06:55:30.542191 28531 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 06:55:30.542447 28531 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:55:30.544070 28531 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 06:55:30.544153 28531 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 06:55:30.544232 28531 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 06:55:30.544281 28531 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 12 06:55:30 dist-test-slave-65mx krb5kdc[28399](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517730, etypes {rep=17 tkt=17 ses=17}, oryx/127.19.52.65@KRBTEST.COM for oryx/127.19.52.126@KRBTEST.COM
I20260812 06:55:30.556227 28531 rpc_server.cc:307] RPC server started. Bound to: 127.19.52.65:34495
I20260812 06:55:30.556339 28675 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.52.65:34495 every 8 connection(s)
I20260812 06:55:30.557549 28531 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786517687384214-19665-0/minicluster-data/ts-0/data/info.pb
I20260812 06:55:30.559854 19665 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu as pid 28531
I20260812 06:55:30.560034 19665 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786517687384214-19665-0/minicluster-data/ts-0/wal/instance
I20260812 06:55:30.560073 28504 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:30.545085 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.52.65:45631 (local address 127.19.52.126:41321)
0812 06:55:30.545251 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0812 06:55:30.545256 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 06:55:30.546203 (+ 947us) server_negotiation.cc:407] Connection header received
0812 06:55:30.547162 (+ 959us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:55:30.547166 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:55:30.547229 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:55:30.547317 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:55:30.549312 (+ 1995us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:30.550218 (+ 906us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:30.551960 (+ 1742us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:30.552222 (+ 262us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:30.552348 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:55:30.555379 (+ 3031us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:55:30.555383 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:55:30.555387 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:55:30.555427 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:55:30.557337 (+ 1910us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:55:30.558059 (+ 722us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:55:30.558066 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:55:30.558069 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:55:30.558190 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:55:30.558777 (+ 587us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:55:30.558781 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:55:30.558783 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:55:30.559015 (+ 232us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:55:30.559136 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0812 06:55:30.559686 (+ 550us) server_negotiation.cc:299] Negotiation successful
0812 06:55:30.559831 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260812 06:55:30.560531 19665 external_mini_cluster.cc:1428] Setting key 4557281d7bd19d47a51fc064d380c851
I20260812 06:55:30.561102 28558 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:30.545534 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.126:41321 (local address 127.19.52.65:45631)
0812 06:55:30.546013 (+ 479us) negotiation.cc:107] Waiting for socket to connect
0812 06:55:30.546065 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0812 06:55:30.546945 (+ 880us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:55:30.547667 (+ 722us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:55:30.547680 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:55:30.548282 (+ 602us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:55:30.549129 (+ 847us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:55:30.549155 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:55:30.550403 (+ 1248us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:55:30.550407 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:55:30.551827 (+ 1420us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:55:30.551838 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:55:30.552498 (+ 660us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:55:30.552510 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:55:30.552731 (+ 221us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:55:30.553410 (+ 679us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 06:55:30.553427 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:55:30.555171 (+ 1744us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 06:55:30.557520 (+ 2349us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:55:30.557525 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:55:30.557538 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:55:30.557933 (+ 395us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:55:30.558322 (+ 389us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:55:30.558327 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:55:30.558332 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:55:30.558649 (+ 317us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:55:30.559136 (+ 487us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 06:55:30.559144 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 06:55:30.559413 (+ 269us) client_negotiation.cc:769] Sending connection context
0812 06:55:30.559641 (+ 228us) client_negotiation.cc:240] Negotiation successful
0812 06:55:30.559989 (+ 348us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":264,"thread_start_us":134,"threads_started":1}
I20260812 06:55:30.562624 28676 heartbeater.cc:347] Connected to a master server at 127.19.52.126:41321
I20260812 06:55:30.565481 28676 heartbeater.cc:464] Registering TS with master...
I20260812 06:55:30.566092 28676 heartbeater.cc:511] Master 127.19.52.126:41321 requested a full tablet report, sending...
I20260812 06:55:30.568791 28449 ts_manager.cc:194] Registered new tserver with Master: 2cebfda268094256bb864f37e363b58c (127.19.52.65:34495)
I20260812 06:55:30.570505 28449 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.19.52.65@KRBTEST.COM'} at 127.19.52.65:45631
I20260812 06:55:30.579090 28504 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:30.566256 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.52.65:43697 (local address 127.19.52.126:41321)
0812 06:55:30.566372 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0812 06:55:30.566376 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 06:55:30.566391 (+ 15us) server_negotiation.cc:407] Connection header received
0812 06:55:30.566445 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:55:30.566448 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:55:30.566487 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:55:30.566585 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:55:30.567159 (+ 574us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:30.567922 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:30.569061 (+ 1139us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:30.569324 (+ 263us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:30.569402 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:55:30.571427 (+ 2025us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:55:30.571475 (+ 48us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:55:30.571485 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:55:30.571523 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:55:30.574101 (+ 2578us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:55:30.574924 (+ 823us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:55:30.574928 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:55:30.574930 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:55:30.575082 (+ 152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:55:30.577659 (+ 2577us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:55:30.577667 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:55:30.577669 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:55:30.578051 (+ 382us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:55:30.578192 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0812 06:55:30.578767 (+ 575us) server_negotiation.cc:299] Negotiation successful
0812 06:55:30.578908 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260812 06:55:30.579113 28683 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:30.565760 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.126:41321 (local address 127.19.52.65:43697)
0812 06:55:30.566073 (+ 313us) negotiation.cc:107] Waiting for socket to connect
0812 06:55:30.566091 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0812 06:55:30.566198 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:55:30.566606 (+ 408us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:55:30.566609 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:55:30.566889 (+ 280us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:55:30.567026 (+ 137us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:55:30.567033 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:55:30.568066 (+ 1033us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:55:30.568069 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:55:30.568884 (+ 815us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:55:30.568924 (+ 40us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:55:30.569618 (+ 694us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:55:30.569624 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:55:30.569747 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:55:30.570364 (+ 617us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 06:55:30.570383 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:55:30.571152 (+ 769us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 06:55:30.574270 (+ 3118us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:55:30.574275 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:55:30.574280 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:55:30.574766 (+ 486us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:55:30.575209 (+ 443us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:55:30.575214 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:55:30.575216 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:55:30.575341 (+ 125us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:55:30.578258 (+ 2917us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 06:55:30.578263 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 06:55:30.578397 (+ 134us) client_negotiation.cc:769] Sending connection context
0812 06:55:30.578701 (+ 304us) client_negotiation.cc:240] Negotiation successful
0812 06:55:30.578901 (+ 200us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":220,"thread_start_us":140,"threads_started":1}
WARNING: no policy specified for oryx/127.19.52.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.19.52.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.19.52.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786517687384214-19665-0/krb5kdc/oryx_127.19.52.66.keytab.
Entry for principal oryx/127.19.52.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786517687384214-19665-0/krb5kdc/oryx_127.19.52.66.keytab.
I20260812 06:55:30.609614 19665 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu
/tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786517687384214-19665-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786517687384214-19665-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786517687384214-19665-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786517687384214-19665-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.52.66:0
--local_ip_for_outbound_sockets=127.19.52.66
--webserver_interface=127.19.52.66
--webserver_port=0
--tserver_master_addrs=127.19.52.126:41321
--keytab_file=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786517687384214-19665-0/krb5kdc/oryx_127.19.52.66.keytab
--principal=oryx/127.19.52.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.52.84:35457
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786517687384214-19665-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786517687384214-19665-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786517687384214-19665-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 06:55:30.737490 28686 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:55:30.737931 28686 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:55:30.738027 28686 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 06:55:30.742334 28686 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 06:55:30.742455 28686 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 06:55:30.742532 28686 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260812 06:55:30.742627 28686 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.52.66
I20260812 06:55:30.748355 28686 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.52.84:35457
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786517687384214-19665-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786517687384214-19665-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786517687384214-19665-0/krb5kdc/oryx_127.19.52.66.keytab
--principal=oryx/127.19.52.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.52.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786517687384214-19665-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.19.52.66
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.19.52.126:41321
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28686
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.52.66
--log_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786517687384214-19665-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 06:15:54 UTC on ccycloud.quasar-rnfzce.root.comops.site
build id 4581989
I20260812 06:55:30.749606 28686 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:55:30.750706 28686 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 06:55:30.761904 28693 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:55:30.761947 28696 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:55:30.762292 28694 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:55:30.762209 28686 server_base.cc:1034] running on GCE node
I20260812 06:55:30.763079 28686 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:55:30.764072 28686 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:55:30.765275 28686 hybrid_clock.cc:648] HybridClock initialized: now 1786517730765227 us; error 65 us; skew 500 ppm
Aug 12 06:55:30 dist-test-slave-65mx krb5kdc[28399](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517730, etypes {rep=17 tkt=17 ses=17}, oryx/127.19.52.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 06:55:30.769083 28686 init.cc:377] Logged in from keytab as oryx/127.19.52.66@KRBTEST.COM (short username oryx)
I20260812 06:55:30.770138 28686 webserver.cc:460] Webserver started at http://127.19.52.66:34883/ using document root <none> and password file <none>
I20260812 06:55:30.770744 28686 fs_manager.cc:359] Metadata directory not provided
I20260812 06:55:30.770813 28686 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:55:30.771051 28686 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 06:55:30.773351 28686 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786517687384214-19665-0/minicluster-data/ts-1/data/instance:
uuid: "b2b871392e9945ef93d6708b940f10e6"
format_stamp: "Formatted at 2026-08-12 06:55:30 on dist-test-slave-65mx"
server_key: "c1e5fc0d43f9b06d30d5aaad37e30983"
server_key_iv: "35679a2ff13d5c7817ae28b97221bd03"
server_key_version: "encryptionkey@0"
I20260812 06:55:30.773976 28686 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786517687384214-19665-0/minicluster-data/ts-1/wal/instance:
uuid: "b2b871392e9945ef93d6708b940f10e6"
format_stamp: "Formatted at 2026-08-12 06:55:30 on dist-test-slave-65mx"
server_key: "c1e5fc0d43f9b06d30d5aaad37e30983"
server_key_iv: "35679a2ff13d5c7817ae28b97221bd03"
server_key_version: "encryptionkey@0"
I20260812 06:55:30.778877 28686 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260812 06:55:30.781673 28704 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 06:55:30.783061 28686 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 06:55:30.783241 28686 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786517687384214-19665-0/minicluster-data/ts-1/data,/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786517687384214-19665-0/minicluster-data/ts-1/wal
uuid: "b2b871392e9945ef93d6708b940f10e6"
format_stamp: "Formatted at 2026-08-12 06:55:30 on dist-test-slave-65mx"
server_key: "c1e5fc0d43f9b06d30d5aaad37e30983"
server_key_iv: "35679a2ff13d5c7817ae28b97221bd03"
server_key_version: "encryptionkey@0"
I20260812 06:55:30.783399 28686 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786517687384214-19665-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786517687384214-19665-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786517687384214-19665-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:55:30.810323 28686 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 06:55:30.813992 28686 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 06:55:30.814236 28686 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:55:30.815752 28686 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 06:55:30.815832 28686 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 06:55:30.815905 28686 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 06:55:30.815958 28686 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 12 06:55:30 dist-test-slave-65mx krb5kdc[28399](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517730, etypes {rep=17 tkt=17 ses=17}, oryx/127.19.52.66@KRBTEST.COM for oryx/127.19.52.126@KRBTEST.COM
I20260812 06:55:30.827544 28686 rpc_server.cc:307] RPC server started. Bound to: 127.19.52.66:38373
I20260812 06:55:30.827631 28839 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.52.66:38373 every 8 connection(s)
I20260812 06:55:30.828492 28686 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786517687384214-19665-0/minicluster-data/ts-1/data/info.pb
I20260812 06:55:30.829684 19665 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu as pid 28686
I20260812 06:55:30.829823 19665 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786517687384214-19665-0/minicluster-data/ts-1/wal/instance
I20260812 06:55:30.830129 19665 external_mini_cluster.cc:1428] Setting key ebcfd62769d39a471aff80871dc923a9
I20260812 06:55:30.831115 28504 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:30.817032 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.52.66:39805 (local address 127.19.52.126:41321)
0812 06:55:30.817215 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0812 06:55:30.817220 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 06:55:30.818205 (+ 985us) server_negotiation.cc:407] Connection header received
0812 06:55:30.819696 (+ 1491us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:55:30.819701 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:55:30.819793 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:55:30.819905 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:55:30.821716 (+ 1811us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:30.822331 (+ 615us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:30.823666 (+ 1335us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:30.823903 (+ 237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:30.824035 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:55:30.826907 (+ 2872us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:55:30.826910 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:55:30.826912 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:55:30.826938 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:55:30.828629 (+ 1691us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:55:30.829264 (+ 635us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:55:30.829268 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:55:30.829270 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:55:30.829362 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:55:30.829762 (+ 400us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:55:30.829767 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:55:30.829769 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:55:30.830040 (+ 271us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:55:30.830167 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0812 06:55:30.830806 (+ 639us) server_negotiation.cc:299] Negotiation successful
0812 06:55:30.830966 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260812 06:55:30.832115 28715 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:30.817456 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.126:41321 (local address 127.19.52.66:39805)
0812 06:55:30.817995 (+ 539us) negotiation.cc:107] Waiting for socket to connect
0812 06:55:30.818060 (+ 65us) client_negotiation.cc:174] Beginning negotiation
0812 06:55:30.819328 (+ 1268us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:55:30.820126 (+ 798us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:55:30.820135 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:55:30.820708 (+ 573us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:55:30.821489 (+ 781us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:55:30.821519 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:55:30.822489 (+ 970us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:55:30.822495 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:55:30.823492 (+ 997us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:55:30.823501 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:55:30.824095 (+ 594us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:55:30.824099 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:55:30.824274 (+ 175us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:55:30.825010 (+ 736us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 06:55:30.825027 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:55:30.826699 (+ 1672us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 06:55:30.828765 (+ 2066us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:55:30.828771 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:55:30.828785 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:55:30.829114 (+ 329us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:55:30.829451 (+ 337us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:55:30.829454 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:55:30.829456 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:55:30.829644 (+ 188us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:55:30.830158 (+ 514us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 06:55:30.830165 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 06:55:30.830434 (+ 269us) client_negotiation.cc:769] Sending connection context
0812 06:55:30.830684 (+ 250us) client_negotiation.cc:240] Negotiation successful
0812 06:55:30.831033 (+ 349us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":302,"thread_start_us":129,"threads_started":1}
I20260812 06:55:30.833459 28840 heartbeater.cc:347] Connected to a master server at 127.19.52.126:41321
I20260812 06:55:30.833654 28840 heartbeater.cc:464] Registering TS with master...
I20260812 06:55:30.834227 28840 heartbeater.cc:511] Master 127.19.52.126:41321 requested a full tablet report, sending...
I20260812 06:55:30.835474 28449 ts_manager.cc:194] Registered new tserver with Master: b2b871392e9945ef93d6708b940f10e6 (127.19.52.66:38373)
I20260812 06:55:30.836395 28449 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.19.52.66@KRBTEST.COM'} at 127.19.52.66:39805
WARNING: no policy specified for oryx/127.19.52.67@KRBTEST.COM; defaulting to no policy
I20260812 06:55:30.845532 28849 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:30.836152 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.126:41321 (local address 127.19.52.66:60469)
0812 06:55:30.837648 (+ 1496us) negotiation.cc:107] Waiting for socket to connect
0812 06:55:30.837667 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0812 06:55:30.837788 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:55:30.838181 (+ 393us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:55:30.838186 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:55:30.838454 (+ 268us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:55:30.838611 (+ 157us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:55:30.838620 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:55:30.839417 (+ 797us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:55:30.839420 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:55:30.840028 (+ 608us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:55:30.840037 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:55:30.840753 (+ 716us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:55:30.840755 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:55:30.840829 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:55:30.841406 (+ 577us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 06:55:30.841418 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:55:30.841769 (+ 351us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 06:55:30.843807 (+ 2038us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:55:30.843810 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:55:30.843811 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:55:30.844022 (+ 211us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:55:30.844358 (+ 336us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:55:30.844360 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:55:30.844361 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:55:30.844441 (+ 80us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:55:30.845034 (+ 593us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 06:55:30.845036 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 06:55:30.845148 (+ 112us) client_negotiation.cc:769] Sending connection context
0812 06:55:30.845238 (+ 90us) client_negotiation.cc:240] Negotiation successful
0812 06:55:30.845357 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1375,"thread_start_us":314,"threads_started":1}
I20260812 06:55:30.845572 28504 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:30.836172 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.52.66:60469 (local address 127.19.52.126:41321)
0812 06:55:30.836393 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0812 06:55:30.836400 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 06:55:30.837760 (+ 1360us) server_negotiation.cc:407] Connection header received
0812 06:55:30.837973 (+ 213us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:55:30.837976 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:55:30.838030 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:55:30.838119 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:55:30.838756 (+ 637us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:30.839296 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:30.840165 (+ 869us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:30.840628 (+ 463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:30.840761 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:55:30.841927 (+ 1166us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:55:30.841931 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:55:30.841934 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:55:30.841967 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:55:30.843672 (+ 1705us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:55:30.844141 (+ 469us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:55:30.844145 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:55:30.844146 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:55:30.844252 (+ 106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:55:30.844536 (+ 284us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:55:30.844539 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:55:30.844540 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:55:30.844879 (+ 339us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:55:30.845034 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0812 06:55:30.845333 (+ 299us) server_negotiation.cc:299] Negotiation successful
0812 06:55:30.845451 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.19.52.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.19.52.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786517687384214-19665-0/krb5kdc/oryx_127.19.52.67.keytab.
Entry for principal oryx/127.19.52.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786517687384214-19665-0/krb5kdc/oryx_127.19.52.67.keytab.
I20260812 06:55:30.868428 19665 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu
/tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786517687384214-19665-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786517687384214-19665-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786517687384214-19665-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786517687384214-19665-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.52.67:0
--local_ip_for_outbound_sockets=127.19.52.67
--webserver_interface=127.19.52.67
--webserver_port=0
--tserver_master_addrs=127.19.52.126:41321
--keytab_file=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786517687384214-19665-0/krb5kdc/oryx_127.19.52.67.keytab
--principal=oryx/127.19.52.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.52.84:35457
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786517687384214-19665-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786517687384214-19665-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786517687384214-19665-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 06:55:30.984880 28851 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:55:30.985173 28851 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:55:30.985221 28851 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 06:55:30.989547 28851 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 06:55:30.989621 28851 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 06:55:30.989662 28851 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260812 06:55:30.989727 28851 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.52.67
I20260812 06:55:30.995252 28851 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.52.84:35457
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786517687384214-19665-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786517687384214-19665-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786517687384214-19665-0/krb5kdc/oryx_127.19.52.67.keytab
--principal=oryx/127.19.52.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.52.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786517687384214-19665-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.19.52.67
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.19.52.126:41321
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28851
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.52.67
--log_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786517687384214-19665-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 06:15:54 UTC on ccycloud.quasar-rnfzce.root.comops.site
build id 4581989
I20260812 06:55:30.996400 28851 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:55:30.997509 28851 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 06:55:31.007373 28859 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:55:31.007488 28857 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:55:31.007589 28851 server_base.cc:1034] running on GCE node
W20260812 06:55:31.007387 28856 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:55:31.008128 28851 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:55:31.008905 28851 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:55:31.010140 28851 hybrid_clock.cc:648] HybridClock initialized: now 1786517731010116 us; error 32 us; skew 500 ppm
Aug 12 06:55:31 dist-test-slave-65mx krb5kdc[28399](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517731, etypes {rep=17 tkt=17 ses=17}, oryx/127.19.52.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 06:55:31.013336 28851 init.cc:377] Logged in from keytab as oryx/127.19.52.67@KRBTEST.COM (short username oryx)
I20260812 06:55:31.014318 28851 webserver.cc:460] Webserver started at http://127.19.52.67:33429/ using document root <none> and password file <none>
I20260812 06:55:31.014982 28851 fs_manager.cc:359] Metadata directory not provided
I20260812 06:55:31.015100 28851 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:55:31.015358 28851 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 06:55:31.017728 28851 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786517687384214-19665-0/minicluster-data/ts-2/data/instance:
uuid: "a2cecf39b4004e88aadc9378c23aa474"
format_stamp: "Formatted at 2026-08-12 06:55:31 on dist-test-slave-65mx"
server_key: "09eb575e07ee0e1fc41c2e6ea00a93d1"
server_key_iv: "25a30da6e7004ba5270ac5e9ddb3e0d5"
server_key_version: "encryptionkey@0"
I20260812 06:55:31.018363 28851 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786517687384214-19665-0/minicluster-data/ts-2/wal/instance:
uuid: "a2cecf39b4004e88aadc9378c23aa474"
format_stamp: "Formatted at 2026-08-12 06:55:31 on dist-test-slave-65mx"
server_key: "09eb575e07ee0e1fc41c2e6ea00a93d1"
server_key_iv: "25a30da6e7004ba5270ac5e9ddb3e0d5"
server_key_version: "encryptionkey@0"
I20260812 06:55:31.022614 28851 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260812 06:55:31.025609 28866 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 06:55:31.027118 28851 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260812 06:55:31.027236 28851 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786517687384214-19665-0/minicluster-data/ts-2/data,/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786517687384214-19665-0/minicluster-data/ts-2/wal
uuid: "a2cecf39b4004e88aadc9378c23aa474"
format_stamp: "Formatted at 2026-08-12 06:55:31 on dist-test-slave-65mx"
server_key: "09eb575e07ee0e1fc41c2e6ea00a93d1"
server_key_iv: "25a30da6e7004ba5270ac5e9ddb3e0d5"
server_key_version: "encryptionkey@0"
I20260812 06:55:31.027405 28851 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786517687384214-19665-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786517687384214-19665-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786517687384214-19665-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:55:31.042092 28851 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 06:55:31.045504 28851 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 06:55:31.045722 28851 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:55:31.047261 28851 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 06:55:31.047338 28851 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 06:55:31.047415 28851 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 06:55:31.047461 28851 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 12 06:55:31 dist-test-slave-65mx krb5kdc[28399](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517731, etypes {rep=17 tkt=17 ses=17}, oryx/127.19.52.67@KRBTEST.COM for oryx/127.19.52.126@KRBTEST.COM
I20260812 06:55:31.060019 28851 rpc_server.cc:307] RPC server started. Bound to: 127.19.52.67:44615
I20260812 06:55:31.060015 28998 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.52.67:44615 every 8 connection(s)
I20260812 06:55:31.061491 28851 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786517687384214-19665-0/minicluster-data/ts-2/data/info.pb
I20260812 06:55:31.064265 28504 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:31.048131 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.52.67:57883 (local address 127.19.52.126:41321)
0812 06:55:31.048284 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0812 06:55:31.048289 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 06:55:31.049251 (+ 962us) server_negotiation.cc:407] Connection header received
0812 06:55:31.050330 (+ 1079us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:55:31.050336 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:55:31.050438 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:55:31.050503 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:55:31.052324 (+ 1821us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:31.053238 (+ 914us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:31.055094 (+ 1856us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:31.055578 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:31.055753 (+ 175us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:55:31.059653 (+ 3900us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:55:31.059668 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:55:31.059676 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:55:31.059719 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:55:31.061692 (+ 1973us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:55:31.062443 (+ 751us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:55:31.062447 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:55:31.062449 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:55:31.062558 (+ 109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:55:31.062945 (+ 387us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:55:31.062948 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:55:31.062949 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:55:31.063152 (+ 203us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:55:31.063236 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0812 06:55:31.063990 (+ 754us) server_negotiation.cc:299] Negotiation successful
0812 06:55:31.064134 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260812 06:55:31.065171 28876 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:31.048607 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.126:41321 (local address 127.19.52.67:57883)
0812 06:55:31.049068 (+ 461us) negotiation.cc:107] Waiting for socket to connect
0812 06:55:31.049120 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0812 06:55:31.050076 (+ 956us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:55:31.050993 (+ 917us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:55:31.051002 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:55:31.051441 (+ 439us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:55:31.052170 (+ 729us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:55:31.052192 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:55:31.053384 (+ 1192us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:55:31.053388 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:55:31.054907 (+ 1519us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:55:31.054943 (+ 36us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:55:31.055792 (+ 849us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:55:31.055797 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:55:31.056034 (+ 237us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:55:31.057007 (+ 973us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 06:55:31.057028 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:55:31.059328 (+ 2300us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 06:55:31.061846 (+ 2518us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:55:31.061851 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:55:31.061863 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:55:31.062311 (+ 448us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:55:31.062718 (+ 407us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:55:31.062720 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:55:31.062722 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:55:31.062865 (+ 143us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:55:31.063280 (+ 415us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 06:55:31.063286 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 06:55:31.063664 (+ 378us) client_negotiation.cc:769] Sending connection context
0812 06:55:31.063848 (+ 184us) client_negotiation.cc:240] Negotiation successful
0812 06:55:31.064153 (+ 305us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":254,"thread_start_us":137,"threads_started":1}
I20260812 06:55:31.066582 28999 heartbeater.cc:347] Connected to a master server at 127.19.52.126:41321
I20260812 06:55:31.066776 28999 heartbeater.cc:464] Registering TS with master...
I20260812 06:55:31.067250 19665 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu as pid 28851
I20260812 06:55:31.067399 28999 heartbeater.cc:511] Master 127.19.52.126:41321 requested a full tablet report, sending...
I20260812 06:55:31.067408 19665 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786517687384214-19665-0/minicluster-data/ts-2/wal/instance
I20260812 06:55:31.067986 19665 external_mini_cluster.cc:1428] Setting key 23c17d742dc42435ee3604448a20b9fb
I20260812 06:55:31.068531 28450 ts_manager.cc:194] Registered new tserver with Master: a2cecf39b4004e88aadc9378c23aa474 (127.19.52.67:44615)
I20260812 06:55:31.069379 28450 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.19.52.67@KRBTEST.COM'} at 127.19.52.67:57883
I20260812 06:55:31.070411 19665 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 06:55:31.078269 28504 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:31.069984 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.52.67:56751 (local address 127.19.52.126:41321)
0812 06:55:31.070107 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0812 06:55:31.070112 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 06:55:31.070129 (+ 17us) server_negotiation.cc:407] Connection header received
0812 06:55:31.070173 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:55:31.070179 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:55:31.070232 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:55:31.070342 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:55:31.070924 (+ 582us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:31.071492 (+ 568us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:31.072431 (+ 939us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:31.072767 (+ 336us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:31.072893 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:55:31.074390 (+ 1497us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:55:31.074402 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:55:31.074405 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:55:31.074442 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:55:31.076195 (+ 1753us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:55:31.076813 (+ 618us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:55:31.076818 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:55:31.076820 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:55:31.076932 (+ 112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:55:31.077278 (+ 346us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:55:31.077283 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:55:31.077285 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:55:31.077507 (+ 222us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:55:31.077635 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0812 06:55:31.077936 (+ 301us) server_negotiation.cc:299] Negotiation successful
0812 06:55:31.078072 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260812 06:55:31.078292 29005 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:31.069613 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.126:41321 (local address 127.19.52.67:56751)
0812 06:55:31.069928 (+ 315us) negotiation.cc:107] Waiting for socket to connect
0812 06:55:31.069943 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 06:55:31.070043 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:55:31.070364 (+ 321us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:55:31.070367 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:55:31.070655 (+ 288us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:55:31.070790 (+ 135us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:55:31.070797 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:55:31.071639 (+ 842us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:55:31.071643 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:55:31.072282 (+ 639us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:55:31.072291 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:55:31.072898 (+ 607us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:55:31.072902 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:55:31.073018 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:55:31.073625 (+ 607us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 06:55:31.073646 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:55:31.074207 (+ 561us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 06:55:31.076340 (+ 2133us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:55:31.076345 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:55:31.076347 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:55:31.076694 (+ 347us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:55:31.077054 (+ 360us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:55:31.077059 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:55:31.077061 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:55:31.077169 (+ 108us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:55:31.077623 (+ 454us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 06:55:31.077627 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 06:55:31.077748 (+ 121us) client_negotiation.cc:769] Sending connection context
0812 06:55:31.077876 (+ 128us) client_negotiation.cc:240] Negotiation successful
0812 06:55:31.078068 (+ 192us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":216,"mutex_wait_us":2,"thread_start_us":120,"threads_started":1}
Aug 12 06:55:31 dist-test-slave-65mx krb5kdc[28399](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517731, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 12 06:55:31 dist-test-slave-65mx krb5kdc[28399](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.19.52.126@KRBTEST.COM, Server not found in Kerberos database
Aug 12 06:55:31 dist-test-slave-65mx krb5kdc[28399](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.19.52.126@KRBTEST.COM, Server not found in Kerberos database
W20260812 06:55:31.099591 28504 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 06:55:31.093028 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42088 (local address 127.19.52.126:41321)
0812 06:55:31.093194 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0812 06:55:31.093198 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 06:55:31.093210 (+ 12us) server_negotiation.cc:407] Connection header received
0812 06:55:31.093251 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:55:31.093253 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:55:31.093294 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:55:31.093366 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:55:31.094092 (+ 726us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:31.094627 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:31.095489 (+ 862us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:31.095696 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:31.095771 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:55:31.099394 (+ 3623us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42088: BlockingRecv error: recv got EOF from 127.0.0.1:42088 (error 108)
Metrics: {"server-negotiator.queue_time_us":58}
Aug 12 06:55:31 dist-test-slave-65mx krb5kdc[28399](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517731, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for oryx/127.19.52.126@KRBTEST.COM
I20260812 06:55:31.109918 28504 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:31.101266 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42096 (local address 127.19.52.126:41321)
0812 06:55:31.101437 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0812 06:55:31.101441 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 06:55:31.101595 (+ 154us) server_negotiation.cc:407] Connection header received
0812 06:55:31.101770 (+ 175us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:55:31.101773 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:55:31.101814 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:55:31.101907 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:55:31.102540 (+ 633us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:31.103088 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:31.103917 (+ 829us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:31.104141 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:31.104252 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:55:31.106819 (+ 2567us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:55:31.106822 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:55:31.106824 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:55:31.106849 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:55:31.108236 (+ 1387us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:55:31.108702 (+ 466us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:55:31.108705 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:55:31.108706 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:55:31.108787 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:55:31.109044 (+ 257us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:55:31.109046 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:55:31.109047 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:55:31.109218 (+ 171us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:55:31.109320 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0812 06:55:31.109582 (+ 262us) server_negotiation.cc:299] Negotiation successful
0812 06:55:31.109739 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260812 06:55:31.113647 28449 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:42096:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 06:55:31.116765 28449 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 06:55:31.131946 29023 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:31.125235 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.65:34495 (local address 127.0.0.1:53332)
0812 06:55:31.125619 (+ 384us) negotiation.cc:107] Waiting for socket to connect
0812 06:55:31.125699 (+ 80us) client_negotiation.cc:174] Beginning negotiation
0812 06:55:31.125843 (+ 144us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:55:31.127375 (+ 1532us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:55:31.127378 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:55:31.127400 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 06:55:31.127531 (+ 131us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:55:31.127536 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:55:31.128731 (+ 1195us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:55:31.128734 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:55:31.130183 (+ 1449us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:55:31.130194 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:55:31.131345 (+ 1151us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:55:31.131349 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:55:31.131479 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:55:31.131537 (+ 58us) client_negotiation.cc:769] Sending connection context
0812 06:55:31.131642 (+ 105us) client_negotiation.cc:240] Negotiation successful
0812 06:55:31.131789 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":197,"mutex_wait_us":22,"thread_start_us":98,"threads_started":1}
I20260812 06:55:31.132284 29028 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:31.125942 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.66:38373 (local address 127.0.0.1:58876)
0812 06:55:31.126195 (+ 253us) negotiation.cc:107] Waiting for socket to connect
0812 06:55:31.126205 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0812 06:55:31.126304 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:55:31.128129 (+ 1825us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:55:31.128132 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:55:31.128151 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 06:55:31.128231 (+ 80us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:55:31.128236 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:55:31.129561 (+ 1325us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:55:31.129565 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:55:31.130682 (+ 1117us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:55:31.130692 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:55:31.131825 (+ 1133us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:55:31.131828 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:55:31.131965 (+ 137us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:55:31.131996 (+ 31us) client_negotiation.cc:769] Sending connection context
0812 06:55:31.132076 (+ 80us) client_negotiation.cc:240] Negotiation successful
0812 06:55:31.132149 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":199,"mutex_wait_us":5,"thread_start_us":112,"threads_started":1}
I20260812 06:55:31.132480 29024 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:31.125386 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53332 (local address 127.19.52.65:34495)
0812 06:55:31.126965 (+ 1579us) server_negotiation.cc:206] Beginning negotiation
0812 06:55:31.126970 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 06:55:31.126990 (+ 20us) server_negotiation.cc:407] Connection header received
0812 06:55:31.127060 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:55:31.127064 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:55:31.127198 (+ 134us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:55:31.127373 (+ 175us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 06:55:31.127656 (+ 283us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:31.128608 (+ 952us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:31.130325 (+ 1717us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:31.131207 (+ 882us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:31.131451 (+ 244us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:55:31.131588 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0812 06:55:31.131677 (+ 89us) server_negotiation.cc:299] Negotiation successful
0812 06:55:31.131860 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1485,"thread_start_us":92,"threads_started":1}
I20260812 06:55:31.132776 29029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:31.126510 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58876 (local address 127.19.52.66:38373)
0812 06:55:31.127687 (+ 1177us) server_negotiation.cc:206] Beginning negotiation
0812 06:55:31.127701 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0812 06:55:31.127721 (+ 20us) server_negotiation.cc:407] Connection header received
0812 06:55:31.127794 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:55:31.127801 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:55:31.127964 (+ 163us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:55:31.128125 (+ 161us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 06:55:31.128356 (+ 231us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:31.129465 (+ 1109us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:31.130817 (+ 1352us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:31.131620 (+ 803us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:31.131783 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:55:31.131926 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0812 06:55:31.132503 (+ 577us) server_negotiation.cc:299] Negotiation successful
0812 06:55:31.132642 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1063,"thread_start_us":128,"threads_started":1}
I20260812 06:55:31.132808 29025 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:31.125621 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.67:44615 (local address 127.0.0.1:59064)
0812 06:55:31.126111 (+ 490us) negotiation.cc:107] Waiting for socket to connect
0812 06:55:31.126125 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 06:55:31.126215 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:55:31.127298 (+ 1083us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:55:31.127306 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:55:31.127334 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 06:55:31.127569 (+ 235us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:55:31.127578 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:55:31.129063 (+ 1485us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:55:31.129066 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:55:31.130174 (+ 1108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:55:31.130189 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:55:31.131331 (+ 1142us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:55:31.131334 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:55:31.132503 (+ 1169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:55:31.132529 (+ 26us) client_negotiation.cc:769] Sending connection context
0812 06:55:31.132619 (+ 90us) client_negotiation.cc:240] Negotiation successful
0812 06:55:31.132683 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":429,"mutex_wait_us":96,"thread_start_us":93,"threads_started":1}
I20260812 06:55:31.133251 29030 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:31.126551 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59064 (local address 127.19.52.67:44615)
0812 06:55:31.126877 (+ 326us) server_negotiation.cc:206] Beginning negotiation
0812 06:55:31.126886 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 06:55:31.126906 (+ 20us) server_negotiation.cc:407] Connection header received
0812 06:55:31.126974 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:55:31.126979 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:55:31.127137 (+ 158us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:55:31.127260 (+ 123us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 06:55:31.127716 (+ 456us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:31.128922 (+ 1206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:31.130340 (+ 1418us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:31.131199 (+ 859us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:31.131330 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:55:31.131506 (+ 176us) server_negotiation.cc:1035] Waiting for connection context
0812 06:55:31.133011 (+ 1505us) server_negotiation.cc:299] Negotiation successful
0812 06:55:31.133116 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":224,"thread_start_us":122,"threads_started":1}
I20260812 06:55:31.135761 28607 tablet_service.cc:1467] Processing CreateTablet for tablet a6a908fc3c90409497468e4afbcad93b (DEFAULT_TABLE table=test-table [id=ba2f759a214e47cfad15b0330b98cb91]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 06:55:31.135766 28922 tablet_service.cc:1467] Processing CreateTablet for tablet a6a908fc3c90409497468e4afbcad93b (DEFAULT_TABLE table=test-table [id=ba2f759a214e47cfad15b0330b98cb91]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 06:55:31.136945 28607 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a6a908fc3c90409497468e4afbcad93b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 06:55:31.136957 28922 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a6a908fc3c90409497468e4afbcad93b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 06:55:31.138449 28764 tablet_service.cc:1467] Processing CreateTablet for tablet a6a908fc3c90409497468e4afbcad93b (DEFAULT_TABLE table=test-table [id=ba2f759a214e47cfad15b0330b98cb91]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 06:55:31.139253 28764 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a6a908fc3c90409497468e4afbcad93b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 06:55:31.144567 29031 tablet_bootstrap.cc:492] T a6a908fc3c90409497468e4afbcad93b P 2cebfda268094256bb864f37e363b58c: Bootstrap starting.
I20260812 06:55:31.145200 29032 tablet_bootstrap.cc:492] T a6a908fc3c90409497468e4afbcad93b P a2cecf39b4004e88aadc9378c23aa474: Bootstrap starting.
I20260812 06:55:31.146955 29031 tablet_bootstrap.cc:654] T a6a908fc3c90409497468e4afbcad93b P 2cebfda268094256bb864f37e363b58c: Neither blocks nor log segments found. Creating new log.
I20260812 06:55:31.147168 29032 tablet_bootstrap.cc:654] T a6a908fc3c90409497468e4afbcad93b P a2cecf39b4004e88aadc9378c23aa474: Neither blocks nor log segments found. Creating new log.
I20260812 06:55:31.147754 29031 log.cc:824] T a6a908fc3c90409497468e4afbcad93b P 2cebfda268094256bb864f37e363b58c: Log is configured to *not* fsync() on all Append() calls
I20260812 06:55:31.147856 29032 log.cc:824] T a6a908fc3c90409497468e4afbcad93b P a2cecf39b4004e88aadc9378c23aa474: Log is configured to *not* fsync() on all Append() calls
I20260812 06:55:31.148447 29033 tablet_bootstrap.cc:492] T a6a908fc3c90409497468e4afbcad93b P b2b871392e9945ef93d6708b940f10e6: Bootstrap starting.
I20260812 06:55:31.150036 29031 tablet_bootstrap.cc:492] T a6a908fc3c90409497468e4afbcad93b P 2cebfda268094256bb864f37e363b58c: No bootstrap required, opened a new log
I20260812 06:55:31.150099 29033 tablet_bootstrap.cc:654] T a6a908fc3c90409497468e4afbcad93b P b2b871392e9945ef93d6708b940f10e6: Neither blocks nor log segments found. Creating new log.
I20260812 06:55:31.150182 29032 tablet_bootstrap.cc:492] T a6a908fc3c90409497468e4afbcad93b P a2cecf39b4004e88aadc9378c23aa474: No bootstrap required, opened a new log
I20260812 06:55:31.150256 29031 ts_tablet_manager.cc:1397] T a6a908fc3c90409497468e4afbcad93b P 2cebfda268094256bb864f37e363b58c: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260812 06:55:31.150436 29032 ts_tablet_manager.cc:1397] T a6a908fc3c90409497468e4afbcad93b P a2cecf39b4004e88aadc9378c23aa474: Time spent bootstrapping tablet: real 0.005s user 0.005s sys 0.000s
I20260812 06:55:31.150701 29033 log.cc:824] T a6a908fc3c90409497468e4afbcad93b P b2b871392e9945ef93d6708b940f10e6: Log is configured to *not* fsync() on all Append() calls
I20260812 06:55:31.152709 29033 tablet_bootstrap.cc:492] T a6a908fc3c90409497468e4afbcad93b P b2b871392e9945ef93d6708b940f10e6: No bootstrap required, opened a new log
I20260812 06:55:31.152866 29033 ts_tablet_manager.cc:1397] T a6a908fc3c90409497468e4afbcad93b P b2b871392e9945ef93d6708b940f10e6: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260812 06:55:31.154783 29032 raft_consensus.cc:348] T a6a908fc3c90409497468e4afbcad93b P a2cecf39b4004e88aadc9378c23aa474 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2cebfda268094256bb864f37e363b58c" member_type: VOTER last_known_addr { host: "127.19.52.65" port: 34495 } } peers { permanent_uuid: "b2b871392e9945ef93d6708b940f10e6" member_type: VOTER last_known_addr { host: "127.19.52.66" port: 38373 } } peers { permanent_uuid: "a2cecf39b4004e88aadc9378c23aa474" member_type: VOTER last_known_addr { host: "127.19.52.67" port: 44615 } }
I20260812 06:55:31.154783 29031 raft_consensus.cc:348] T a6a908fc3c90409497468e4afbcad93b P 2cebfda268094256bb864f37e363b58c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2cebfda268094256bb864f37e363b58c" member_type: VOTER last_known_addr { host: "127.19.52.65" port: 34495 } } peers { permanent_uuid: "b2b871392e9945ef93d6708b940f10e6" member_type: VOTER last_known_addr { host: "127.19.52.66" port: 38373 } } peers { permanent_uuid: "a2cecf39b4004e88aadc9378c23aa474" member_type: VOTER last_known_addr { host: "127.19.52.67" port: 44615 } }
I20260812 06:55:31.155093 29031 raft_consensus.cc:374] T a6a908fc3c90409497468e4afbcad93b P 2cebfda268094256bb864f37e363b58c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 06:55:31.155151 29031 raft_consensus.cc:729] T a6a908fc3c90409497468e4afbcad93b P 2cebfda268094256bb864f37e363b58c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2cebfda268094256bb864f37e363b58c, State: Initialized, Role: FOLLOWER
I20260812 06:55:31.155262 29032 raft_consensus.cc:374] T a6a908fc3c90409497468e4afbcad93b P a2cecf39b4004e88aadc9378c23aa474 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 06:55:31.155364 29032 raft_consensus.cc:729] T a6a908fc3c90409497468e4afbcad93b P a2cecf39b4004e88aadc9378c23aa474 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a2cecf39b4004e88aadc9378c23aa474, State: Initialized, Role: FOLLOWER
I20260812 06:55:31.155649 29031 consensus_queue.cc:260] T a6a908fc3c90409497468e4afbcad93b P 2cebfda268094256bb864f37e363b58c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2cebfda268094256bb864f37e363b58c" member_type: VOTER last_known_addr { host: "127.19.52.65" port: 34495 } } peers { permanent_uuid: "b2b871392e9945ef93d6708b940f10e6" member_type: VOTER last_known_addr { host: "127.19.52.66" port: 38373 } } peers { permanent_uuid: "a2cecf39b4004e88aadc9378c23aa474" member_type: VOTER last_known_addr { host: "127.19.52.67" port: 44615 } }
I20260812 06:55:31.155921 29032 consensus_queue.cc:260] T a6a908fc3c90409497468e4afbcad93b P a2cecf39b4004e88aadc9378c23aa474 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2cebfda268094256bb864f37e363b58c" member_type: VOTER last_known_addr { host: "127.19.52.65" port: 34495 } } peers { permanent_uuid: "b2b871392e9945ef93d6708b940f10e6" member_type: VOTER last_known_addr { host: "127.19.52.66" port: 38373 } } peers { permanent_uuid: "a2cecf39b4004e88aadc9378c23aa474" member_type: VOTER last_known_addr { host: "127.19.52.67" port: 44615 } }
I20260812 06:55:31.155853 29033 raft_consensus.cc:348] T a6a908fc3c90409497468e4afbcad93b P b2b871392e9945ef93d6708b940f10e6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2cebfda268094256bb864f37e363b58c" member_type: VOTER last_known_addr { host: "127.19.52.65" port: 34495 } } peers { permanent_uuid: "b2b871392e9945ef93d6708b940f10e6" member_type: VOTER last_known_addr { host: "127.19.52.66" port: 38373 } } peers { permanent_uuid: "a2cecf39b4004e88aadc9378c23aa474" member_type: VOTER last_known_addr { host: "127.19.52.67" port: 44615 } }
I20260812 06:55:31.156121 29033 raft_consensus.cc:374] T a6a908fc3c90409497468e4afbcad93b P b2b871392e9945ef93d6708b940f10e6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 06:55:31.156193 29033 raft_consensus.cc:729] T a6a908fc3c90409497468e4afbcad93b P b2b871392e9945ef93d6708b940f10e6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b2b871392e9945ef93d6708b940f10e6, State: Initialized, Role: FOLLOWER
I20260812 06:55:31.156425 28676 heartbeater.cc:503] Master 127.19.52.126:41321 was elected leader, sending a full tablet report...
I20260812 06:55:31.156674 29033 consensus_queue.cc:260] T a6a908fc3c90409497468e4afbcad93b P b2b871392e9945ef93d6708b940f10e6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2cebfda268094256bb864f37e363b58c" member_type: VOTER last_known_addr { host: "127.19.52.65" port: 34495 } } peers { permanent_uuid: "b2b871392e9945ef93d6708b940f10e6" member_type: VOTER last_known_addr { host: "127.19.52.66" port: 38373 } } peers { permanent_uuid: "a2cecf39b4004e88aadc9378c23aa474" member_type: VOTER last_known_addr { host: "127.19.52.67" port: 44615 } }
I20260812 06:55:31.156981 28999 heartbeater.cc:503] Master 127.19.52.126:41321 was elected leader, sending a full tablet report...
I20260812 06:55:31.157394 29032 ts_tablet_manager.cc:1428] T a6a908fc3c90409497468e4afbcad93b P a2cecf39b4004e88aadc9378c23aa474: Time spent starting tablet: real 0.007s user 0.006s sys 0.000s
I20260812 06:55:31.157670 29033 ts_tablet_manager.cc:1428] T a6a908fc3c90409497468e4afbcad93b P b2b871392e9945ef93d6708b940f10e6: Time spent starting tablet: real 0.005s user 0.004s sys 0.000s
I20260812 06:55:31.158111 28840 heartbeater.cc:503] Master 127.19.52.126:41321 was elected leader, sending a full tablet report...
I20260812 06:55:31.159636 29031 ts_tablet_manager.cc:1428] T a6a908fc3c90409497468e4afbcad93b P 2cebfda268094256bb864f37e363b58c: Time spent starting tablet: real 0.009s user 0.008s sys 0.000s
I20260812 06:55:31.187146 29042 raft_consensus.cc:482] T a6a908fc3c90409497468e4afbcad93b P b2b871392e9945ef93d6708b940f10e6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 06:55:31.187399 29042 raft_consensus.cc:504] T a6a908fc3c90409497468e4afbcad93b P b2b871392e9945ef93d6708b940f10e6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2cebfda268094256bb864f37e363b58c" member_type: VOTER last_known_addr { host: "127.19.52.65" port: 34495 } } peers { permanent_uuid: "b2b871392e9945ef93d6708b940f10e6" member_type: VOTER last_known_addr { host: "127.19.52.66" port: 38373 } } peers { permanent_uuid: "a2cecf39b4004e88aadc9378c23aa474" member_type: VOTER last_known_addr { host: "127.19.52.67" port: 44615 } }
I20260812 06:55:31.188803 29042 leader_election.cc:288] T a6a908fc3c90409497468e4afbcad93b P b2b871392e9945ef93d6708b940f10e6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2cebfda268094256bb864f37e363b58c (127.19.52.65:34495), a2cecf39b4004e88aadc9378c23aa474 (127.19.52.67:44615)
I20260812 06:55:31.193478 29030 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:31.189210 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.52.66:34033 (local address 127.19.52.67:44615)
0812 06:55:31.189337 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0812 06:55:31.189342 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 06:55:31.189425 (+ 83us) server_negotiation.cc:407] Connection header received
0812 06:55:31.189636 (+ 211us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:55:31.189640 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:55:31.189702 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:55:31.189790 (+ 88us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 06:55:31.190179 (+ 389us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:31.190937 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:31.192285 (+ 1348us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:31.192853 (+ 568us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:31.193034 (+ 181us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:55:31.193090 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0812 06:55:31.193280 (+ 190us) server_negotiation.cc:299] Negotiation successful
0812 06:55:31.193352 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260812 06:55:31.193477 28715 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:31.189162 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.67:44615 (local address 127.19.52.66:34033)
0812 06:55:31.189317 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0812 06:55:31.189335 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0812 06:55:31.189484 (+ 149us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:55:31.189831 (+ 347us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:55:31.189834 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:55:31.189880 (+ 46us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 06:55:31.190032 (+ 152us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:55:31.190040 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:55:31.191055 (+ 1015us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:55:31.191058 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:55:31.192154 (+ 1096us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:55:31.192164 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:55:31.193024 (+ 860us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:55:31.193028 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:55:31.193139 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:55:31.193169 (+ 30us) client_negotiation.cc:769] Sending connection context
0812 06:55:31.193264 (+ 95us) client_negotiation.cc:240] Negotiation successful
0812 06:55:31.193339 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":43}
I20260812 06:55:31.194123 28945 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a6a908fc3c90409497468e4afbcad93b" candidate_uuid: "b2b871392e9945ef93d6708b940f10e6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2cecf39b4004e88aadc9378c23aa474" is_pre_election: true
I20260812 06:55:31.194484 29044 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:31.189165 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.65:34495 (local address 127.19.52.66:50971)
0812 06:55:31.190045 (+ 880us) negotiation.cc:107] Waiting for socket to connect
0812 06:55:31.190059 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 06:55:31.190166 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:55:31.190442 (+ 276us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:55:31.190445 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:55:31.190478 (+ 33us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 06:55:31.191046 (+ 568us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:55:31.191054 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:55:31.191935 (+ 881us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:55:31.191942 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:55:31.193028 (+ 1086us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:55:31.193039 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:55:31.194052 (+ 1013us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:55:31.194055 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:55:31.194132 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:55:31.194150 (+ 18us) client_negotiation.cc:769] Sending connection context
0812 06:55:31.194216 (+ 66us) client_negotiation.cc:240] Negotiation successful
0812 06:55:31.194303 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":811,"mutex_wait_us":449,"thread_start_us":171,"threads_started":1}
I20260812 06:55:31.194491 29024 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:31.189273 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.52.66:50971 (local address 127.19.52.65:34495)
0812 06:55:31.189421 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0812 06:55:31.189430 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 06:55:31.190163 (+ 733us) server_negotiation.cc:407] Connection header received
0812 06:55:31.190255 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:55:31.190258 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:55:31.190313 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:55:31.190399 (+ 86us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 06:55:31.191184 (+ 785us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:31.191799 (+ 615us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:31.193179 (+ 1380us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:31.193783 (+ 604us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:31.193922 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:55:31.193971 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0812 06:55:31.194281 (+ 310us) server_negotiation.cc:299] Negotiation successful
0812 06:55:31.194367 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260812 06:55:31.194476 28945 raft_consensus.cc:2454] T a6a908fc3c90409497468e4afbcad93b P a2cecf39b4004e88aadc9378c23aa474 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b2b871392e9945ef93d6708b940f10e6 in term 0.
I20260812 06:55:31.195050 28709 leader_election.cc:302] T a6a908fc3c90409497468e4afbcad93b P b2b871392e9945ef93d6708b940f10e6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2cecf39b4004e88aadc9378c23aa474, b2b871392e9945ef93d6708b940f10e6; no voters:
I20260812 06:55:31.195150 28628 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a6a908fc3c90409497468e4afbcad93b" candidate_uuid: "b2b871392e9945ef93d6708b940f10e6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2cebfda268094256bb864f37e363b58c" is_pre_election: true
I20260812 06:55:31.195370 29042 raft_consensus.cc:2781] T a6a908fc3c90409497468e4afbcad93b P b2b871392e9945ef93d6708b940f10e6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 06:55:31.195484 29042 raft_consensus.cc:482] T a6a908fc3c90409497468e4afbcad93b P b2b871392e9945ef93d6708b940f10e6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 06:55:31.195466 28628 raft_consensus.cc:2454] T a6a908fc3c90409497468e4afbcad93b P 2cebfda268094256bb864f37e363b58c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b2b871392e9945ef93d6708b940f10e6 in term 0.
I20260812 06:55:31.195544 29042 raft_consensus.cc:3037] T a6a908fc3c90409497468e4afbcad93b P b2b871392e9945ef93d6708b940f10e6 [term 0 FOLLOWER]: Advancing to term 1
I20260812 06:55:31.196765 29042 raft_consensus.cc:504] T a6a908fc3c90409497468e4afbcad93b P b2b871392e9945ef93d6708b940f10e6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2cebfda268094256bb864f37e363b58c" member_type: VOTER last_known_addr { host: "127.19.52.65" port: 34495 } } peers { permanent_uuid: "b2b871392e9945ef93d6708b940f10e6" member_type: VOTER last_known_addr { host: "127.19.52.66" port: 38373 } } peers { permanent_uuid: "a2cecf39b4004e88aadc9378c23aa474" member_type: VOTER last_known_addr { host: "127.19.52.67" port: 44615 } }
I20260812 06:55:31.197175 29042 leader_election.cc:288] T a6a908fc3c90409497468e4afbcad93b P b2b871392e9945ef93d6708b940f10e6 [CANDIDATE]: Term 1 election: Requested vote from peers 2cebfda268094256bb864f37e363b58c (127.19.52.65:34495), a2cecf39b4004e88aadc9378c23aa474 (127.19.52.67:44615)
I20260812 06:55:31.197739 28945 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a6a908fc3c90409497468e4afbcad93b" candidate_uuid: "b2b871392e9945ef93d6708b940f10e6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2cecf39b4004e88aadc9378c23aa474"
I20260812 06:55:31.197852 28628 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a6a908fc3c90409497468e4afbcad93b" candidate_uuid: "b2b871392e9945ef93d6708b940f10e6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2cebfda268094256bb864f37e363b58c"
I20260812 06:55:31.197886 28945 raft_consensus.cc:3037] T a6a908fc3c90409497468e4afbcad93b P a2cecf39b4004e88aadc9378c23aa474 [term 0 FOLLOWER]: Advancing to term 1
I20260812 06:55:31.197988 28628 raft_consensus.cc:3037] T a6a908fc3c90409497468e4afbcad93b P 2cebfda268094256bb864f37e363b58c [term 0 FOLLOWER]: Advancing to term 1
I20260812 06:55:31.199302 28628 raft_consensus.cc:2454] T a6a908fc3c90409497468e4afbcad93b P 2cebfda268094256bb864f37e363b58c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b2b871392e9945ef93d6708b940f10e6 in term 1.
I20260812 06:55:31.199302 28945 raft_consensus.cc:2454] T a6a908fc3c90409497468e4afbcad93b P a2cecf39b4004e88aadc9378c23aa474 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b2b871392e9945ef93d6708b940f10e6 in term 1.
I20260812 06:55:31.199707 28707 leader_election.cc:302] T a6a908fc3c90409497468e4afbcad93b P b2b871392e9945ef93d6708b940f10e6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2cebfda268094256bb864f37e363b58c, b2b871392e9945ef93d6708b940f10e6; no voters:
I20260812 06:55:31.199929 29042 raft_consensus.cc:2781] T a6a908fc3c90409497468e4afbcad93b P b2b871392e9945ef93d6708b940f10e6 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 06:55:31.200263 29042 raft_consensus.cc:686] T a6a908fc3c90409497468e4afbcad93b P b2b871392e9945ef93d6708b940f10e6 [term 1 LEADER]: Becoming Leader. State: Replica: b2b871392e9945ef93d6708b940f10e6, State: Running, Role: LEADER
I20260812 06:55:31.200738 29042 consensus_queue.cc:237] T a6a908fc3c90409497468e4afbcad93b P b2b871392e9945ef93d6708b940f10e6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2cebfda268094256bb864f37e363b58c" member_type: VOTER last_known_addr { host: "127.19.52.65" port: 34495 } } peers { permanent_uuid: "b2b871392e9945ef93d6708b940f10e6" member_type: VOTER last_known_addr { host: "127.19.52.66" port: 38373 } } peers { permanent_uuid: "a2cecf39b4004e88aadc9378c23aa474" member_type: VOTER last_known_addr { host: "127.19.52.67" port: 44615 } }
I20260812 06:55:31.203943 28449 catalog_manager.cc:5495] T a6a908fc3c90409497468e4afbcad93b P b2b871392e9945ef93d6708b940f10e6 reported cstate change: term changed from 0 to 1, leader changed from <none> to b2b871392e9945ef93d6708b940f10e6 (127.19.52.66). New cstate: current_term: 1 leader_uuid: "b2b871392e9945ef93d6708b940f10e6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2cebfda268094256bb864f37e363b58c" member_type: VOTER last_known_addr { host: "127.19.52.65" port: 34495 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b2b871392e9945ef93d6708b940f10e6" member_type: VOTER last_known_addr { host: "127.19.52.66" port: 38373 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a2cecf39b4004e88aadc9378c23aa474" member_type: VOTER last_known_addr { host: "127.19.52.67" port: 44615 } health_report { overall_health: UNKNOWN } } }
I20260812 06:55:31.216651 29029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:31.213095 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58884 (local address 127.19.52.66:38373)
0812 06:55:31.213233 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0812 06:55:31.213238 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 06:55:31.213253 (+ 15us) server_negotiation.cc:407] Connection header received
0812 06:55:31.213368 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:55:31.213371 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:55:31.213423 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:55:31.213547 (+ 124us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 06:55:31.213784 (+ 237us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:31.214293 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:31.215152 (+ 859us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:31.215374 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:31.215494 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:55:31.215768 (+ 274us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 06:55:31.216187 (+ 419us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 06:55:31.216322 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0812 06:55:31.216385 (+ 63us) server_negotiation.cc:299] Negotiation successful
0812 06:55:31.216444 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260812 06:55:31.222617 28628 raft_consensus.cc:1261] T a6a908fc3c90409497468e4afbcad93b P 2cebfda268094256bb864f37e363b58c [term 1 FOLLOWER]: Refusing update from remote peer b2b871392e9945ef93d6708b940f10e6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 06:55:31.222617 28945 raft_consensus.cc:1261] T a6a908fc3c90409497468e4afbcad93b P a2cecf39b4004e88aadc9378c23aa474 [term 1 FOLLOWER]: Refusing update from remote peer b2b871392e9945ef93d6708b940f10e6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 06:55:31.223469 29042 consensus_queue.cc:1035] T a6a908fc3c90409497468e4afbcad93b P b2b871392e9945ef93d6708b940f10e6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2cecf39b4004e88aadc9378c23aa474" member_type: VOTER last_known_addr { host: "127.19.52.67" port: 44615 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 06:55:31.223711 29045 consensus_queue.cc:1035] T a6a908fc3c90409497468e4afbcad93b P b2b871392e9945ef93d6708b940f10e6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2cebfda268094256bb864f37e363b58c" member_type: VOTER last_known_addr { host: "127.19.52.65" port: 34495 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 06:55:31.231135 29048 mvcc.cc:205] Tried to move back new op lower bound from 7317576627079704576 to 7317576627000950784. Current Snapshot: MvccSnapshot[applied={T|T < 7317576627079704576}]
I20260812 06:55:31.232666 29050 mvcc.cc:205] Tried to move back new op lower bound from 7317576627079704576 to 7317576627000950784. Current Snapshot: MvccSnapshot[applied={T|T < 7317576627079704576}]
I20260812 06:55:31.240749 28449 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:42096:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 06:55:31.240934 28449 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:42096:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 06:55:31.243603 28449 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 3bce2c30f41c4196af2accc6509b44ff: Sending DeleteTablet for 3 replicas of tablet a6a908fc3c90409497468e4afbcad93b
I20260812 06:55:31.244490 28607 tablet_service.cc:1514] Processing DeleteTablet for tablet a6a908fc3c90409497468e4afbcad93b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 06:55:31 UTC) from {username='oryx', principal='oryx/127.19.52.126@KRBTEST.COM'} at 127.0.0.1:53332
I20260812 06:55:31.244606 28922 tablet_service.cc:1514] Processing DeleteTablet for tablet a6a908fc3c90409497468e4afbcad93b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 06:55:31 UTC) from {username='oryx', principal='oryx/127.19.52.126@KRBTEST.COM'} at 127.0.0.1:59064
I20260812 06:55:31.245086 28764 tablet_service.cc:1514] Processing DeleteTablet for tablet a6a908fc3c90409497468e4afbcad93b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 06:55:31 UTC) from {username='oryx', principal='oryx/127.19.52.126@KRBTEST.COM'} at 127.0.0.1:58876
I20260812 06:55:31.245604 29062 tablet_replica.cc:330] T a6a908fc3c90409497468e4afbcad93b P b2b871392e9945ef93d6708b940f10e6: stopping tablet replica
I20260812 06:55:31.245967 29062 raft_consensus.cc:2229] T a6a908fc3c90409497468e4afbcad93b P b2b871392e9945ef93d6708b940f10e6 [term 1 LEADER]: Raft consensus shutting down.
I20260812 06:55:31.246374 29062 raft_consensus.cc:2258] T a6a908fc3c90409497468e4afbcad93b P b2b871392e9945ef93d6708b940f10e6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 06:55:31.246551 29061 tablet_replica.cc:330] T a6a908fc3c90409497468e4afbcad93b P a2cecf39b4004e88aadc9378c23aa474: stopping tablet replica
I20260812 06:55:31.246493 28448 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:42096:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 06:55:31.246927 28448 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 06:55:31.246830 29061 raft_consensus.cc:2229] T a6a908fc3c90409497468e4afbcad93b P a2cecf39b4004e88aadc9378c23aa474 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 06:55:31.247267 29061 raft_consensus.cc:2258] T a6a908fc3c90409497468e4afbcad93b P a2cecf39b4004e88aadc9378c23aa474 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 06:55:31.248314 29062 ts_tablet_manager.cc:1905] T a6a908fc3c90409497468e4afbcad93b P b2b871392e9945ef93d6708b940f10e6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 06:55:31.248728 29061 ts_tablet_manager.cc:1905] T a6a908fc3c90409497468e4afbcad93b P a2cecf39b4004e88aadc9378c23aa474: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 06:55:31.251356 29062 ts_tablet_manager.cc:1918] T a6a908fc3c90409497468e4afbcad93b P b2b871392e9945ef93d6708b940f10e6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 06:55:31.251531 29060 tablet_replica.cc:330] T a6a908fc3c90409497468e4afbcad93b P 2cebfda268094256bb864f37e363b58c: stopping tablet replica
I20260812 06:55:31.251565 29062 log.cc:1196] T a6a908fc3c90409497468e4afbcad93b P b2b871392e9945ef93d6708b940f10e6: Deleting WAL directory at /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786517687384214-19665-0/minicluster-data/ts-1/wal/wals/a6a908fc3c90409497468e4afbcad93b
I20260812 06:55:31.251868 29060 raft_consensus.cc:2229] T a6a908fc3c90409497468e4afbcad93b P 2cebfda268094256bb864f37e363b58c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 06:55:31.252123 29062 ts_tablet_manager.cc:1939] T a6a908fc3c90409497468e4afbcad93b P b2b871392e9945ef93d6708b940f10e6: Deleting consensus metadata
I20260812 06:55:31.252203 29060 raft_consensus.cc:2258] T a6a908fc3c90409497468e4afbcad93b P 2cebfda268094256bb864f37e363b58c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 06:55:31.252785 29061 ts_tablet_manager.cc:1918] T a6a908fc3c90409497468e4afbcad93b P a2cecf39b4004e88aadc9378c23aa474: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 06:55:31.252939 29061 log.cc:1196] T a6a908fc3c90409497468e4afbcad93b P a2cecf39b4004e88aadc9378c23aa474: Deleting WAL directory at /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786517687384214-19665-0/minicluster-data/ts-2/wal/wals/a6a908fc3c90409497468e4afbcad93b
I20260812 06:55:31.253327 28922 tablet_service.cc:1467] Processing CreateTablet for tablet 4183186e9bd24a8a861a59b01884aa93 (DEFAULT_TABLE table=test-table [id=f8bc3c9400f8435ab1dc221b8cf49dd2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 06:55:31.253764 28922 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4183186e9bd24a8a861a59b01884aa93. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 06:55:31.253965 28607 tablet_service.cc:1467] Processing CreateTablet for tablet 4183186e9bd24a8a861a59b01884aa93 (DEFAULT_TABLE table=test-table [id=f8bc3c9400f8435ab1dc221b8cf49dd2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 06:55:31.254124 28764 tablet_service.cc:1467] Processing CreateTablet for tablet 4183186e9bd24a8a861a59b01884aa93 (DEFAULT_TABLE table=test-table [id=f8bc3c9400f8435ab1dc221b8cf49dd2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 06:55:31.254235 28607 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4183186e9bd24a8a861a59b01884aa93. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 06:55:31.254377 28764 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4183186e9bd24a8a861a59b01884aa93. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 06:55:31.253314 29061 ts_tablet_manager.cc:1939] T a6a908fc3c90409497468e4afbcad93b P a2cecf39b4004e88aadc9378c23aa474: Deleting consensus metadata
I20260812 06:55:31.256841 29031 tablet_bootstrap.cc:492] T 4183186e9bd24a8a861a59b01884aa93 P 2cebfda268094256bb864f37e363b58c: Bootstrap starting.
I20260812 06:55:31.256861 29032 tablet_bootstrap.cc:492] T 4183186e9bd24a8a861a59b01884aa93 P a2cecf39b4004e88aadc9378c23aa474: Bootstrap starting.
I20260812 06:55:31.257567 29033 tablet_bootstrap.cc:492] T 4183186e9bd24a8a861a59b01884aa93 P b2b871392e9945ef93d6708b940f10e6: Bootstrap starting.
I20260812 06:55:31.257881 29031 tablet_bootstrap.cc:654] T 4183186e9bd24a8a861a59b01884aa93 P 2cebfda268094256bb864f37e363b58c: Neither blocks nor log segments found. Creating new log.
I20260812 06:55:31.257918 29032 tablet_bootstrap.cc:654] T 4183186e9bd24a8a861a59b01884aa93 P a2cecf39b4004e88aadc9378c23aa474: Neither blocks nor log segments found. Creating new log.
I20260812 06:55:31.258615 29033 tablet_bootstrap.cc:654] T 4183186e9bd24a8a861a59b01884aa93 P b2b871392e9945ef93d6708b940f10e6: Neither blocks nor log segments found. Creating new log.
I20260812 06:55:31.259361 29031 tablet_bootstrap.cc:492] T 4183186e9bd24a8a861a59b01884aa93 P 2cebfda268094256bb864f37e363b58c: No bootstrap required, opened a new log
I20260812 06:55:31.259459 29031 ts_tablet_manager.cc:1397] T 4183186e9bd24a8a861a59b01884aa93 P 2cebfda268094256bb864f37e363b58c: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260812 06:55:31.259805 28434 catalog_manager.cc:4841] TS a2cecf39b4004e88aadc9378c23aa474 (127.19.52.67:44615): tablet a6a908fc3c90409497468e4afbcad93b (table test-table [id=ba2f759a214e47cfad15b0330b98cb91]) successfully deleted
I20260812 06:55:31.259898 29033 tablet_bootstrap.cc:492] T 4183186e9bd24a8a861a59b01884aa93 P b2b871392e9945ef93d6708b940f10e6: No bootstrap required, opened a new log
I20260812 06:55:31.259982 29033 ts_tablet_manager.cc:1397] T 4183186e9bd24a8a861a59b01884aa93 P b2b871392e9945ef93d6708b940f10e6: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260812 06:55:31.259956 29031 raft_consensus.cc:348] T 4183186e9bd24a8a861a59b01884aa93 P 2cebfda268094256bb864f37e363b58c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2cebfda268094256bb864f37e363b58c" member_type: VOTER last_known_addr { host: "127.19.52.65" port: 34495 } } peers { permanent_uuid: "b2b871392e9945ef93d6708b940f10e6" member_type: VOTER last_known_addr { host: "127.19.52.66" port: 38373 } } peers { permanent_uuid: "a2cecf39b4004e88aadc9378c23aa474" member_type: VOTER last_known_addr { host: "127.19.52.67" port: 44615 } }
I20260812 06:55:31.260073 29031 raft_consensus.cc:374] T 4183186e9bd24a8a861a59b01884aa93 P 2cebfda268094256bb864f37e363b58c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 06:55:31.260113 29031 raft_consensus.cc:729] T 4183186e9bd24a8a861a59b01884aa93 P 2cebfda268094256bb864f37e363b58c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2cebfda268094256bb864f37e363b58c, State: Initialized, Role: FOLLOWER
I20260812 06:55:31.260243 29031 consensus_queue.cc:260] T 4183186e9bd24a8a861a59b01884aa93 P 2cebfda268094256bb864f37e363b58c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2cebfda268094256bb864f37e363b58c" member_type: VOTER last_known_addr { host: "127.19.52.65" port: 34495 } } peers { permanent_uuid: "b2b871392e9945ef93d6708b940f10e6" member_type: VOTER last_known_addr { host: "127.19.52.66" port: 38373 } } peers { permanent_uuid: "a2cecf39b4004e88aadc9378c23aa474" member_type: VOTER last_known_addr { host: "127.19.52.67" port: 44615 } }
I20260812 06:55:31.260480 29033 raft_consensus.cc:348] T 4183186e9bd24a8a861a59b01884aa93 P b2b871392e9945ef93d6708b940f10e6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2cebfda268094256bb864f37e363b58c" member_type: VOTER last_known_addr { host: "127.19.52.65" port: 34495 } } peers { permanent_uuid: "b2b871392e9945ef93d6708b940f10e6" member_type: VOTER last_known_addr { host: "127.19.52.66" port: 38373 } } peers { permanent_uuid: "a2cecf39b4004e88aadc9378c23aa474" member_type: VOTER last_known_addr { host: "127.19.52.67" port: 44615 } }
I20260812 06:55:31.260630 29033 raft_consensus.cc:374] T 4183186e9bd24a8a861a59b01884aa93 P b2b871392e9945ef93d6708b940f10e6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 06:55:31.260675 29033 raft_consensus.cc:729] T 4183186e9bd24a8a861a59b01884aa93 P b2b871392e9945ef93d6708b940f10e6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b2b871392e9945ef93d6708b940f10e6, State: Initialized, Role: FOLLOWER
I20260812 06:55:31.260748 29032 tablet_bootstrap.cc:492] T 4183186e9bd24a8a861a59b01884aa93 P a2cecf39b4004e88aadc9378c23aa474: No bootstrap required, opened a new log
I20260812 06:55:31.260826 29032 ts_tablet_manager.cc:1397] T 4183186e9bd24a8a861a59b01884aa93 P a2cecf39b4004e88aadc9378c23aa474: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260812 06:55:31.260818 29033 consensus_queue.cc:260] T 4183186e9bd24a8a861a59b01884aa93 P b2b871392e9945ef93d6708b940f10e6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2cebfda268094256bb864f37e363b58c" member_type: VOTER last_known_addr { host: "127.19.52.65" port: 34495 } } peers { permanent_uuid: "b2b871392e9945ef93d6708b940f10e6" member_type: VOTER last_known_addr { host: "127.19.52.66" port: 38373 } } peers { permanent_uuid: "a2cecf39b4004e88aadc9378c23aa474" member_type: VOTER last_known_addr { host: "127.19.52.67" port: 44615 } }
I20260812 06:55:31.261248 29032 raft_consensus.cc:348] T 4183186e9bd24a8a861a59b01884aa93 P a2cecf39b4004e88aadc9378c23aa474 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2cebfda268094256bb864f37e363b58c" member_type: VOTER last_known_addr { host: "127.19.52.65" port: 34495 } } peers { permanent_uuid: "b2b871392e9945ef93d6708b940f10e6" member_type: VOTER last_known_addr { host: "127.19.52.66" port: 38373 } } peers { permanent_uuid: "a2cecf39b4004e88aadc9378c23aa474" member_type: VOTER last_known_addr { host: "127.19.52.67" port: 44615 } }
I20260812 06:55:31.261371 29032 raft_consensus.cc:374] T 4183186e9bd24a8a861a59b01884aa93 P a2cecf39b4004e88aadc9378c23aa474 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 06:55:31.261428 29032 raft_consensus.cc:729] T 4183186e9bd24a8a861a59b01884aa93 P a2cecf39b4004e88aadc9378c23aa474 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a2cecf39b4004e88aadc9378c23aa474, State: Initialized, Role: FOLLOWER
I20260812 06:55:31.261561 29032 consensus_queue.cc:260] T 4183186e9bd24a8a861a59b01884aa93 P a2cecf39b4004e88aadc9378c23aa474 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2cebfda268094256bb864f37e363b58c" member_type: VOTER last_known_addr { host: "127.19.52.65" port: 34495 } } peers { permanent_uuid: "b2b871392e9945ef93d6708b940f10e6" member_type: VOTER last_known_addr { host: "127.19.52.66" port: 38373 } } peers { permanent_uuid: "a2cecf39b4004e88aadc9378c23aa474" member_type: VOTER last_known_addr { host: "127.19.52.67" port: 44615 } }
I20260812 06:55:31.261603 29033 ts_tablet_manager.cc:1428] T 4183186e9bd24a8a861a59b01884aa93 P b2b871392e9945ef93d6708b940f10e6: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260812 06:55:31.261811 29032 ts_tablet_manager.cc:1428] T 4183186e9bd24a8a861a59b01884aa93 P a2cecf39b4004e88aadc9378c23aa474: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260812 06:55:31.264623 29031 ts_tablet_manager.cc:1428] T 4183186e9bd24a8a861a59b01884aa93 P 2cebfda268094256bb864f37e363b58c: Time spent starting tablet: real 0.005s user 0.001s sys 0.000s
I20260812 06:55:31.267678 28433 catalog_manager.cc:4841] TS b2b871392e9945ef93d6708b940f10e6 (127.19.52.66:38373): tablet a6a908fc3c90409497468e4afbcad93b (table test-table [id=ba2f759a214e47cfad15b0330b98cb91]) successfully deleted
I20260812 06:55:31.267911 29060 ts_tablet_manager.cc:1905] T a6a908fc3c90409497468e4afbcad93b P 2cebfda268094256bb864f37e363b58c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 06:55:31.271379 29060 ts_tablet_manager.cc:1918] T a6a908fc3c90409497468e4afbcad93b P 2cebfda268094256bb864f37e363b58c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 06:55:31.271536 29060 log.cc:1196] T a6a908fc3c90409497468e4afbcad93b P 2cebfda268094256bb864f37e363b58c: Deleting WAL directory at /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786517687384214-19665-0/minicluster-data/ts-0/wal/wals/a6a908fc3c90409497468e4afbcad93b
I20260812 06:55:31.271890 29060 ts_tablet_manager.cc:1939] T a6a908fc3c90409497468e4afbcad93b P 2cebfda268094256bb864f37e363b58c: Deleting consensus metadata
I20260812 06:55:31.273277 28433 catalog_manager.cc:4841] TS 2cebfda268094256bb864f37e363b58c (127.19.52.65:34495): tablet a6a908fc3c90409497468e4afbcad93b (table test-table [id=ba2f759a214e47cfad15b0330b98cb91]) successfully deleted
I20260812 06:55:31.283397 29041 raft_consensus.cc:482] T 4183186e9bd24a8a861a59b01884aa93 P a2cecf39b4004e88aadc9378c23aa474 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 06:55:31.283551 29041 raft_consensus.cc:504] T 4183186e9bd24a8a861a59b01884aa93 P a2cecf39b4004e88aadc9378c23aa474 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2cebfda268094256bb864f37e363b58c" member_type: VOTER last_known_addr { host: "127.19.52.65" port: 34495 } } peers { permanent_uuid: "b2b871392e9945ef93d6708b940f10e6" member_type: VOTER last_known_addr { host: "127.19.52.66" port: 38373 } } peers { permanent_uuid: "a2cecf39b4004e88aadc9378c23aa474" member_type: VOTER last_known_addr { host: "127.19.52.67" port: 44615 } }
I20260812 06:55:31.284863 29041 leader_election.cc:288] T 4183186e9bd24a8a861a59b01884aa93 P a2cecf39b4004e88aadc9378c23aa474 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2cebfda268094256bb864f37e363b58c (127.19.52.65:34495), b2b871392e9945ef93d6708b940f10e6 (127.19.52.66:38373)
I20260812 06:55:31.290200 28876 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:31.285112 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.65:34495 (local address 127.19.52.67:43461)
0812 06:55:31.285283 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0812 06:55:31.285304 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0812 06:55:31.285414 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:55:31.285850 (+ 436us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:55:31.285856 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:55:31.285883 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 06:55:31.286040 (+ 157us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:55:31.286052 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:55:31.287218 (+ 1166us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:55:31.287226 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:55:31.288716 (+ 1490us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:55:31.288729 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:55:31.289694 (+ 965us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:55:31.289700 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:55:31.289810 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:55:31.289841 (+ 31us) client_negotiation.cc:769] Sending connection context
0812 06:55:31.289925 (+ 84us) client_negotiation.cc:240] Negotiation successful
0812 06:55:31.290020 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":70,"mutex_wait_us":24}
I20260812 06:55:31.290393 29029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:31.285702 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.52.67:34517 (local address 127.19.52.66:38373)
0812 06:55:31.285844 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0812 06:55:31.285848 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 06:55:31.285907 (+ 59us) server_negotiation.cc:407] Connection header received
0812 06:55:31.286111 (+ 204us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:55:31.286118 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:55:31.286193 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:55:31.286249 (+ 56us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 06:55:31.286732 (+ 483us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:31.287514 (+ 782us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:31.289114 (+ 1600us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:31.289632 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:31.289761 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:55:31.289821 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0812 06:55:31.290156 (+ 335us) server_negotiation.cc:299] Negotiation successful
0812 06:55:31.290263 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260812 06:55:31.290786 29068 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:31.285338 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.66:38373 (local address 127.19.52.67:34517)
0812 06:55:31.285808 (+ 470us) negotiation.cc:107] Waiting for socket to connect
0812 06:55:31.285824 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0812 06:55:31.285971 (+ 147us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:55:31.286389 (+ 418us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:55:31.286392 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:55:31.286412 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 06:55:31.286549 (+ 137us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:55:31.286555 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:55:31.287666 (+ 1111us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:55:31.287670 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:55:31.288985 (+ 1315us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:55:31.288993 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:55:31.289925 (+ 932us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:55:31.289929 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:55:31.290025 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:55:31.290046 (+ 21us) client_negotiation.cc:769] Sending connection context
0812 06:55:31.290116 (+ 70us) client_negotiation.cc:240] Negotiation successful
0812 06:55:31.290173 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":396,"mutex_wait_us":25,"thread_start_us":86,"threads_started":1}
I20260812 06:55:31.290805 29024 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:31.285228 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.52.67:43461 (local address 127.19.52.65:34495)
0812 06:55:31.285394 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0812 06:55:31.285400 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 06:55:31.285417 (+ 17us) server_negotiation.cc:407] Connection header received
0812 06:55:31.285570 (+ 153us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:55:31.285573 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:55:31.285636 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:55:31.285714 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 06:55:31.286190 (+ 476us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:31.287100 (+ 910us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:31.288860 (+ 1760us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:31.289482 (+ 622us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:31.289636 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:55:31.289693 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0812 06:55:31.290551 (+ 858us) server_negotiation.cc:299] Negotiation successful
0812 06:55:31.290679 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260812 06:55:31.291275 28628 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4183186e9bd24a8a861a59b01884aa93" candidate_uuid: "a2cecf39b4004e88aadc9378c23aa474" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2cebfda268094256bb864f37e363b58c" is_pre_election: true
I20260812 06:55:31.291502 28628 raft_consensus.cc:2454] T 4183186e9bd24a8a861a59b01884aa93 P 2cebfda268094256bb864f37e363b58c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2cecf39b4004e88aadc9378c23aa474 in term 0.
I20260812 06:55:31.291621 28788 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4183186e9bd24a8a861a59b01884aa93" candidate_uuid: "a2cecf39b4004e88aadc9378c23aa474" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b2b871392e9945ef93d6708b940f10e6" is_pre_election: true
I20260812 06:55:31.291864 28788 raft_consensus.cc:2454] T 4183186e9bd24a8a861a59b01884aa93 P b2b871392e9945ef93d6708b940f10e6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2cecf39b4004e88aadc9378c23aa474 in term 0.
I20260812 06:55:31.292033 28869 leader_election.cc:302] T 4183186e9bd24a8a861a59b01884aa93 P a2cecf39b4004e88aadc9378c23aa474 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2cebfda268094256bb864f37e363b58c, a2cecf39b4004e88aadc9378c23aa474; no voters:
I20260812 06:55:31.292392 29041 raft_consensus.cc:2781] T 4183186e9bd24a8a861a59b01884aa93 P a2cecf39b4004e88aadc9378c23aa474 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 06:55:31.292632 29041 raft_consensus.cc:482] T 4183186e9bd24a8a861a59b01884aa93 P a2cecf39b4004e88aadc9378c23aa474 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 06:55:31.292702 29041 raft_consensus.cc:3037] T 4183186e9bd24a8a861a59b01884aa93 P a2cecf39b4004e88aadc9378c23aa474 [term 0 FOLLOWER]: Advancing to term 1
I20260812 06:55:31.294414 29041 raft_consensus.cc:504] T 4183186e9bd24a8a861a59b01884aa93 P a2cecf39b4004e88aadc9378c23aa474 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2cebfda268094256bb864f37e363b58c" member_type: VOTER last_known_addr { host: "127.19.52.65" port: 34495 } } peers { permanent_uuid: "b2b871392e9945ef93d6708b940f10e6" member_type: VOTER last_known_addr { host: "127.19.52.66" port: 38373 } } peers { permanent_uuid: "a2cecf39b4004e88aadc9378c23aa474" member_type: VOTER last_known_addr { host: "127.19.52.67" port: 44615 } }
I20260812 06:55:31.294853 29041 leader_election.cc:288] T 4183186e9bd24a8a861a59b01884aa93 P a2cecf39b4004e88aadc9378c23aa474 [CANDIDATE]: Term 1 election: Requested vote from peers 2cebfda268094256bb864f37e363b58c (127.19.52.65:34495), b2b871392e9945ef93d6708b940f10e6 (127.19.52.66:38373)
I20260812 06:55:31.295265 28628 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4183186e9bd24a8a861a59b01884aa93" candidate_uuid: "a2cecf39b4004e88aadc9378c23aa474" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2cebfda268094256bb864f37e363b58c"
I20260812 06:55:31.295327 28788 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4183186e9bd24a8a861a59b01884aa93" candidate_uuid: "a2cecf39b4004e88aadc9378c23aa474" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b2b871392e9945ef93d6708b940f10e6"
I20260812 06:55:31.295430 28628 raft_consensus.cc:3037] T 4183186e9bd24a8a861a59b01884aa93 P 2cebfda268094256bb864f37e363b58c [term 0 FOLLOWER]: Advancing to term 1
I20260812 06:55:31.295463 28788 raft_consensus.cc:3037] T 4183186e9bd24a8a861a59b01884aa93 P b2b871392e9945ef93d6708b940f10e6 [term 0 FOLLOWER]: Advancing to term 1
I20260812 06:55:31.296406 28628 raft_consensus.cc:2454] T 4183186e9bd24a8a861a59b01884aa93 P 2cebfda268094256bb864f37e363b58c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2cecf39b4004e88aadc9378c23aa474 in term 1.
I20260812 06:55:31.296741 28788 raft_consensus.cc:2454] T 4183186e9bd24a8a861a59b01884aa93 P b2b871392e9945ef93d6708b940f10e6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2cecf39b4004e88aadc9378c23aa474 in term 1.
I20260812 06:55:31.296758 28869 leader_election.cc:302] T 4183186e9bd24a8a861a59b01884aa93 P a2cecf39b4004e88aadc9378c23aa474 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2cebfda268094256bb864f37e363b58c, a2cecf39b4004e88aadc9378c23aa474; no voters:
I20260812 06:55:31.296980 29041 raft_consensus.cc:2781] T 4183186e9bd24a8a861a59b01884aa93 P a2cecf39b4004e88aadc9378c23aa474 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 06:55:31.297211 29041 raft_consensus.cc:686] T 4183186e9bd24a8a861a59b01884aa93 P a2cecf39b4004e88aadc9378c23aa474 [term 1 LEADER]: Becoming Leader. State: Replica: a2cecf39b4004e88aadc9378c23aa474, State: Running, Role: LEADER
I20260812 06:55:31.297477 29041 consensus_queue.cc:237] T 4183186e9bd24a8a861a59b01884aa93 P a2cecf39b4004e88aadc9378c23aa474 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2cebfda268094256bb864f37e363b58c" member_type: VOTER last_known_addr { host: "127.19.52.65" port: 34495 } } peers { permanent_uuid: "b2b871392e9945ef93d6708b940f10e6" member_type: VOTER last_known_addr { host: "127.19.52.66" port: 38373 } } peers { permanent_uuid: "a2cecf39b4004e88aadc9378c23aa474" member_type: VOTER last_known_addr { host: "127.19.52.67" port: 44615 } }
I20260812 06:55:31.299880 28449 catalog_manager.cc:5495] T 4183186e9bd24a8a861a59b01884aa93 P a2cecf39b4004e88aadc9378c23aa474 reported cstate change: term changed from 0 to 1, leader changed from <none> to a2cecf39b4004e88aadc9378c23aa474 (127.19.52.67). New cstate: current_term: 1 leader_uuid: "a2cecf39b4004e88aadc9378c23aa474" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2cebfda268094256bb864f37e363b58c" member_type: VOTER last_known_addr { host: "127.19.52.65" port: 34495 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b2b871392e9945ef93d6708b940f10e6" member_type: VOTER last_known_addr { host: "127.19.52.66" port: 38373 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a2cecf39b4004e88aadc9378c23aa474" member_type: VOTER last_known_addr { host: "127.19.52.67" port: 44615 } health_report { overall_health: HEALTHY } } }
W20260812 06:55:31.309280 28677 tablet.cc:2369] T 4183186e9bd24a8a861a59b01884aa93 P 2cebfda268094256bb864f37e363b58c: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 06:55:31.313110 29000 tablet.cc:2369] T 4183186e9bd24a8a861a59b01884aa93 P a2cecf39b4004e88aadc9378c23aa474: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 06:55:31.329962 28842 tablet.cc:2369] T 4183186e9bd24a8a861a59b01884aa93 P b2b871392e9945ef93d6708b940f10e6: Can't schedule compaction. Clean time has not been advanced past its initial value.
Aug 12 06:55:31 dist-test-slave-65mx krb5kdc[28399](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517730, etypes {rep=17 tkt=17 ses=17}, oryx/127.19.52.126@KRBTEST.COM for oryx/127.19.52.126@KRBTEST.COM
I20260812 06:55:31.352294 29077 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:31.342408 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.126:41321 (local address 127.0.0.1:42106)
0812 06:55:31.342707 (+ 299us) negotiation.cc:107] Waiting for socket to connect
0812 06:55:31.342725 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0812 06:55:31.342841 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:55:31.343155 (+ 314us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:55:31.343157 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:55:31.343578 (+ 421us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:55:31.343687 (+ 109us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:55:31.343694 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:55:31.344803 (+ 1109us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:55:31.344805 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:55:31.345394 (+ 589us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:55:31.345403 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:55:31.346191 (+ 788us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:55:31.346194 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:55:31.346258 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:55:31.346981 (+ 723us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 06:55:31.346994 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:55:31.348751 (+ 1757us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 06:55:31.350656 (+ 1905us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:55:31.350665 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:55:31.350670 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:55:31.350918 (+ 248us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:55:31.351226 (+ 308us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:55:31.351228 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:55:31.351229 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:55:31.351311 (+ 82us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:55:31.351785 (+ 474us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 06:55:31.351799 (+ 14us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 06:55:31.351907 (+ 108us) client_negotiation.cc:769] Sending connection context
0812 06:55:31.352017 (+ 110us) client_negotiation.cc:240] Negotiation successful
0812 06:55:31.352150 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":208,"thread_start_us":120,"threads_started":1}
I20260812 06:55:31.352418 28504 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:31.342504 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42106 (local address 127.19.52.126:41321)
0812 06:55:31.342656 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0812 06:55:31.342661 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 06:55:31.342819 (+ 158us) server_negotiation.cc:407] Connection header received
0812 06:55:31.342933 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:55:31.342937 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:55:31.342990 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:55:31.343092 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:55:31.343808 (+ 716us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:31.344672 (+ 864us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:31.345518 (+ 846us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:31.345896 (+ 378us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:31.346012 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:55:31.348904 (+ 2892us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:55:31.348908 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:55:31.348911 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:55:31.348937 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:55:31.350515 (+ 1578us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:55:31.351031 (+ 516us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:55:31.351034 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:55:31.351036 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:55:31.351128 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:55:31.351406 (+ 278us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:55:31.351408 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:55:31.351410 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:55:31.351641 (+ 231us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:55:31.351742 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0812 06:55:31.352109 (+ 367us) server_negotiation.cc:299] Negotiation successful
0812 06:55:31.352234 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260812 06:55:31.357419 28449 catalog_manager.cc:1908] Servicing CreateTable request from {username='oryx', principal='oryx/127.19.52.126@KRBTEST.COM'} at 127.0.0.1:42106:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260812 06:55:31.358429 28449 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 06:55:31.364183 28764 tablet_service.cc:1467] Processing CreateTablet for tablet ab96689a1eb640efac4c74fe5d8dfca8 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=a795e27386ea4ee186c16fc735e0fa99]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260812 06:55:31.364183 28607 tablet_service.cc:1467] Processing CreateTablet for tablet ab96689a1eb640efac4c74fe5d8dfca8 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=a795e27386ea4ee186c16fc735e0fa99]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260812 06:55:31.364372 28922 tablet_service.cc:1467] Processing CreateTablet for tablet ab96689a1eb640efac4c74fe5d8dfca8 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=a795e27386ea4ee186c16fc735e0fa99]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260812 06:55:31.364670 28607 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ab96689a1eb640efac4c74fe5d8dfca8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 06:55:31.364670 28764 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ab96689a1eb640efac4c74fe5d8dfca8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 06:55:31.364816 28922 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ab96689a1eb640efac4c74fe5d8dfca8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 06:55:31.367950 29031 tablet_bootstrap.cc:492] T ab96689a1eb640efac4c74fe5d8dfca8 P 2cebfda268094256bb864f37e363b58c: Bootstrap starting.
I20260812 06:55:31.367950 29033 tablet_bootstrap.cc:492] T ab96689a1eb640efac4c74fe5d8dfca8 P b2b871392e9945ef93d6708b940f10e6: Bootstrap starting.
I20260812 06:55:31.368187 29032 tablet_bootstrap.cc:492] T ab96689a1eb640efac4c74fe5d8dfca8 P a2cecf39b4004e88aadc9378c23aa474: Bootstrap starting.
I20260812 06:55:31.369099 29031 tablet_bootstrap.cc:654] T ab96689a1eb640efac4c74fe5d8dfca8 P 2cebfda268094256bb864f37e363b58c: Neither blocks nor log segments found. Creating new log.
I20260812 06:55:31.369099 29033 tablet_bootstrap.cc:654] T ab96689a1eb640efac4c74fe5d8dfca8 P b2b871392e9945ef93d6708b940f10e6: Neither blocks nor log segments found. Creating new log.
I20260812 06:55:31.369220 29032 tablet_bootstrap.cc:654] T ab96689a1eb640efac4c74fe5d8dfca8 P a2cecf39b4004e88aadc9378c23aa474: Neither blocks nor log segments found. Creating new log.
I20260812 06:55:31.370956 29032 tablet_bootstrap.cc:492] T ab96689a1eb640efac4c74fe5d8dfca8 P a2cecf39b4004e88aadc9378c23aa474: No bootstrap required, opened a new log
I20260812 06:55:31.370990 29033 tablet_bootstrap.cc:492] T ab96689a1eb640efac4c74fe5d8dfca8 P b2b871392e9945ef93d6708b940f10e6: No bootstrap required, opened a new log
I20260812 06:55:31.371059 29033 ts_tablet_manager.cc:1397] T ab96689a1eb640efac4c74fe5d8dfca8 P b2b871392e9945ef93d6708b940f10e6: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.002s
I20260812 06:55:31.371059 29032 ts_tablet_manager.cc:1397] T ab96689a1eb640efac4c74fe5d8dfca8 P a2cecf39b4004e88aadc9378c23aa474: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260812 06:55:31.371583 29033 raft_consensus.cc:348] T ab96689a1eb640efac4c74fe5d8dfca8 P b2b871392e9945ef93d6708b940f10e6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2cebfda268094256bb864f37e363b58c" member_type: VOTER last_known_addr { host: "127.19.52.65" port: 34495 } } peers { permanent_uuid: "b2b871392e9945ef93d6708b940f10e6" member_type: VOTER last_known_addr { host: "127.19.52.66" port: 38373 } } peers { permanent_uuid: "a2cecf39b4004e88aadc9378c23aa474" member_type: VOTER last_known_addr { host: "127.19.52.67" port: 44615 } }
I20260812 06:55:31.371708 29033 raft_consensus.cc:374] T ab96689a1eb640efac4c74fe5d8dfca8 P b2b871392e9945ef93d6708b940f10e6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 06:55:31.371743 29033 raft_consensus.cc:729] T ab96689a1eb640efac4c74fe5d8dfca8 P b2b871392e9945ef93d6708b940f10e6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b2b871392e9945ef93d6708b940f10e6, State: Initialized, Role: FOLLOWER
I20260812 06:55:31.371809 29032 raft_consensus.cc:348] T ab96689a1eb640efac4c74fe5d8dfca8 P a2cecf39b4004e88aadc9378c23aa474 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2cebfda268094256bb864f37e363b58c" member_type: VOTER last_known_addr { host: "127.19.52.65" port: 34495 } } peers { permanent_uuid: "b2b871392e9945ef93d6708b940f10e6" member_type: VOTER last_known_addr { host: "127.19.52.66" port: 38373 } } peers { permanent_uuid: "a2cecf39b4004e88aadc9378c23aa474" member_type: VOTER last_known_addr { host: "127.19.52.67" port: 44615 } }
I20260812 06:55:31.371925 29032 raft_consensus.cc:374] T ab96689a1eb640efac4c74fe5d8dfca8 P a2cecf39b4004e88aadc9378c23aa474 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 06:55:31.371886 29033 consensus_queue.cc:260] T ab96689a1eb640efac4c74fe5d8dfca8 P b2b871392e9945ef93d6708b940f10e6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2cebfda268094256bb864f37e363b58c" member_type: VOTER last_known_addr { host: "127.19.52.65" port: 34495 } } peers { permanent_uuid: "b2b871392e9945ef93d6708b940f10e6" member_type: VOTER last_known_addr { host: "127.19.52.66" port: 38373 } } peers { permanent_uuid: "a2cecf39b4004e88aadc9378c23aa474" member_type: VOTER last_known_addr { host: "127.19.52.67" port: 44615 } }
I20260812 06:55:31.371958 29032 raft_consensus.cc:729] T ab96689a1eb640efac4c74fe5d8dfca8 P a2cecf39b4004e88aadc9378c23aa474 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a2cecf39b4004e88aadc9378c23aa474, State: Initialized, Role: FOLLOWER
I20260812 06:55:31.372074 29032 consensus_queue.cc:260] T ab96689a1eb640efac4c74fe5d8dfca8 P a2cecf39b4004e88aadc9378c23aa474 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2cebfda268094256bb864f37e363b58c" member_type: VOTER last_known_addr { host: "127.19.52.65" port: 34495 } } peers { permanent_uuid: "b2b871392e9945ef93d6708b940f10e6" member_type: VOTER last_known_addr { host: "127.19.52.66" port: 38373 } } peers { permanent_uuid: "a2cecf39b4004e88aadc9378c23aa474" member_type: VOTER last_known_addr { host: "127.19.52.67" port: 44615 } }
I20260812 06:55:31.372191 29042 tablet_replica.cc:439] T ab96689a1eb640efac4c74fe5d8dfca8 P b2b871392e9945ef93d6708b940f10e6: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2cebfda268094256bb864f37e363b58c" member_type: VOTER last_known_addr { host: "127.19.52.65" port: 34495 } } peers { permanent_uuid: "b2b871392e9945ef93d6708b940f10e6" member_type: VOTER last_known_addr { host: "127.19.52.66" port: 38373 } } peers { permanent_uuid: "a2cecf39b4004e88aadc9378c23aa474" member_type: VOTER last_known_addr { host: "127.19.52.67" port: 44615 } } }
I20260812 06:55:31.372362 29042 tablet_replica.cc:442] T ab96689a1eb640efac4c74fe5d8dfca8 P b2b871392e9945ef93d6708b940f10e6: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 06:55:31.372717 29041 tablet_replica.cc:439] T ab96689a1eb640efac4c74fe5d8dfca8 P a2cecf39b4004e88aadc9378c23aa474: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2cebfda268094256bb864f37e363b58c" member_type: VOTER last_known_addr { host: "127.19.52.65" port: 34495 } } peers { permanent_uuid: "b2b871392e9945ef93d6708b940f10e6" member_type: VOTER last_known_addr { host: "127.19.52.66" port: 38373 } } peers { permanent_uuid: "a2cecf39b4004e88aadc9378c23aa474" member_type: VOTER last_known_addr { host: "127.19.52.67" port: 44615 } } }
I20260812 06:55:31.372804 29032 ts_tablet_manager.cc:1428] T ab96689a1eb640efac4c74fe5d8dfca8 P a2cecf39b4004e88aadc9378c23aa474: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260812 06:55:31.372881 29041 tablet_replica.cc:442] T ab96689a1eb640efac4c74fe5d8dfca8 P a2cecf39b4004e88aadc9378c23aa474: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 06:55:31.372200 29033 ts_tablet_manager.cc:1428] T ab96689a1eb640efac4c74fe5d8dfca8 P b2b871392e9945ef93d6708b940f10e6: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260812 06:55:31.373875 29031 tablet_bootstrap.cc:492] T ab96689a1eb640efac4c74fe5d8dfca8 P 2cebfda268094256bb864f37e363b58c: No bootstrap required, opened a new log
I20260812 06:55:31.374006 29031 ts_tablet_manager.cc:1397] T ab96689a1eb640efac4c74fe5d8dfca8 P 2cebfda268094256bb864f37e363b58c: Time spent bootstrapping tablet: real 0.006s user 0.002s sys 0.000s
I20260812 06:55:31.374414 29031 raft_consensus.cc:348] T ab96689a1eb640efac4c74fe5d8dfca8 P 2cebfda268094256bb864f37e363b58c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2cebfda268094256bb864f37e363b58c" member_type: VOTER last_known_addr { host: "127.19.52.65" port: 34495 } } peers { permanent_uuid: "b2b871392e9945ef93d6708b940f10e6" member_type: VOTER last_known_addr { host: "127.19.52.66" port: 38373 } } peers { permanent_uuid: "a2cecf39b4004e88aadc9378c23aa474" member_type: VOTER last_known_addr { host: "127.19.52.67" port: 44615 } }
I20260812 06:55:31.374545 29031 raft_consensus.cc:374] T ab96689a1eb640efac4c74fe5d8dfca8 P 2cebfda268094256bb864f37e363b58c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 06:55:31.374588 29031 raft_consensus.cc:729] T ab96689a1eb640efac4c74fe5d8dfca8 P 2cebfda268094256bb864f37e363b58c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2cebfda268094256bb864f37e363b58c, State: Initialized, Role: FOLLOWER
I20260812 06:55:31.374739 29031 consensus_queue.cc:260] T ab96689a1eb640efac4c74fe5d8dfca8 P 2cebfda268094256bb864f37e363b58c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2cebfda268094256bb864f37e363b58c" member_type: VOTER last_known_addr { host: "127.19.52.65" port: 34495 } } peers { permanent_uuid: "b2b871392e9945ef93d6708b940f10e6" member_type: VOTER last_known_addr { host: "127.19.52.66" port: 38373 } } peers { permanent_uuid: "a2cecf39b4004e88aadc9378c23aa474" member_type: VOTER last_known_addr { host: "127.19.52.67" port: 44615 } }
I20260812 06:55:31.375059 29031 ts_tablet_manager.cc:1428] T ab96689a1eb640efac4c74fe5d8dfca8 P 2cebfda268094256bb864f37e363b58c: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20260812 06:55:31.375278 29039 tablet_replica.cc:439] T ab96689a1eb640efac4c74fe5d8dfca8 P 2cebfda268094256bb864f37e363b58c: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2cebfda268094256bb864f37e363b58c" member_type: VOTER last_known_addr { host: "127.19.52.65" port: 34495 } } peers { permanent_uuid: "b2b871392e9945ef93d6708b940f10e6" member_type: VOTER last_known_addr { host: "127.19.52.66" port: 38373 } } peers { permanent_uuid: "a2cecf39b4004e88aadc9378c23aa474" member_type: VOTER last_known_addr { host: "127.19.52.67" port: 44615 } } }
I20260812 06:55:31.375430 29039 tablet_replica.cc:442] T ab96689a1eb640efac4c74fe5d8dfca8 P 2cebfda268094256bb864f37e363b58c: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 06:55:31.642627 29042 raft_consensus.cc:482] T ab96689a1eb640efac4c74fe5d8dfca8 P b2b871392e9945ef93d6708b940f10e6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 06:55:31.642791 29042 raft_consensus.cc:504] T ab96689a1eb640efac4c74fe5d8dfca8 P b2b871392e9945ef93d6708b940f10e6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2cebfda268094256bb864f37e363b58c" member_type: VOTER last_known_addr { host: "127.19.52.65" port: 34495 } } peers { permanent_uuid: "b2b871392e9945ef93d6708b940f10e6" member_type: VOTER last_known_addr { host: "127.19.52.66" port: 38373 } } peers { permanent_uuid: "a2cecf39b4004e88aadc9378c23aa474" member_type: VOTER last_known_addr { host: "127.19.52.67" port: 44615 } }
I20260812 06:55:31.643338 29042 leader_election.cc:288] T ab96689a1eb640efac4c74fe5d8dfca8 P b2b871392e9945ef93d6708b940f10e6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2cebfda268094256bb864f37e363b58c (127.19.52.65:34495), a2cecf39b4004e88aadc9378c23aa474 (127.19.52.67:44615)
I20260812 06:55:31.643970 28945 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ab96689a1eb640efac4c74fe5d8dfca8" candidate_uuid: "b2b871392e9945ef93d6708b940f10e6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2cecf39b4004e88aadc9378c23aa474" is_pre_election: true
I20260812 06:55:31.644004 28628 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ab96689a1eb640efac4c74fe5d8dfca8" candidate_uuid: "b2b871392e9945ef93d6708b940f10e6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2cebfda268094256bb864f37e363b58c" is_pre_election: true
I20260812 06:55:31.644167 28945 raft_consensus.cc:2454] T ab96689a1eb640efac4c74fe5d8dfca8 P a2cecf39b4004e88aadc9378c23aa474 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b2b871392e9945ef93d6708b940f10e6 in term 0.
I20260812 06:55:31.644192 28628 raft_consensus.cc:2454] T ab96689a1eb640efac4c74fe5d8dfca8 P 2cebfda268094256bb864f37e363b58c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b2b871392e9945ef93d6708b940f10e6 in term 0.
I20260812 06:55:31.644611 28709 leader_election.cc:302] T ab96689a1eb640efac4c74fe5d8dfca8 P b2b871392e9945ef93d6708b940f10e6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2cecf39b4004e88aadc9378c23aa474, b2b871392e9945ef93d6708b940f10e6; no voters:
I20260812 06:55:31.644840 29042 raft_consensus.cc:2781] T ab96689a1eb640efac4c74fe5d8dfca8 P b2b871392e9945ef93d6708b940f10e6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 06:55:31.644912 29042 raft_consensus.cc:482] T ab96689a1eb640efac4c74fe5d8dfca8 P b2b871392e9945ef93d6708b940f10e6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 06:55:31.644944 29042 raft_consensus.cc:3037] T ab96689a1eb640efac4c74fe5d8dfca8 P b2b871392e9945ef93d6708b940f10e6 [term 0 FOLLOWER]: Advancing to term 1
I20260812 06:55:31.646076 29042 raft_consensus.cc:504] T ab96689a1eb640efac4c74fe5d8dfca8 P b2b871392e9945ef93d6708b940f10e6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2cebfda268094256bb864f37e363b58c" member_type: VOTER last_known_addr { host: "127.19.52.65" port: 34495 } } peers { permanent_uuid: "b2b871392e9945ef93d6708b940f10e6" member_type: VOTER last_known_addr { host: "127.19.52.66" port: 38373 } } peers { permanent_uuid: "a2cecf39b4004e88aadc9378c23aa474" member_type: VOTER last_known_addr { host: "127.19.52.67" port: 44615 } }
I20260812 06:55:31.646509 29042 leader_election.cc:288] T ab96689a1eb640efac4c74fe5d8dfca8 P b2b871392e9945ef93d6708b940f10e6 [CANDIDATE]: Term 1 election: Requested vote from peers 2cebfda268094256bb864f37e363b58c (127.19.52.65:34495), a2cecf39b4004e88aadc9378c23aa474 (127.19.52.67:44615)
I20260812 06:55:31.646931 28945 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ab96689a1eb640efac4c74fe5d8dfca8" candidate_uuid: "b2b871392e9945ef93d6708b940f10e6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2cecf39b4004e88aadc9378c23aa474"
I20260812 06:55:31.647055 28945 raft_consensus.cc:3037] T ab96689a1eb640efac4c74fe5d8dfca8 P a2cecf39b4004e88aadc9378c23aa474 [term 0 FOLLOWER]: Advancing to term 1
I20260812 06:55:31.646974 28628 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ab96689a1eb640efac4c74fe5d8dfca8" candidate_uuid: "b2b871392e9945ef93d6708b940f10e6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2cebfda268094256bb864f37e363b58c"
I20260812 06:55:31.647140 28628 raft_consensus.cc:3037] T ab96689a1eb640efac4c74fe5d8dfca8 P 2cebfda268094256bb864f37e363b58c [term 0 FOLLOWER]: Advancing to term 1
I20260812 06:55:31.648090 28628 raft_consensus.cc:2454] T ab96689a1eb640efac4c74fe5d8dfca8 P 2cebfda268094256bb864f37e363b58c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b2b871392e9945ef93d6708b940f10e6 in term 1.
I20260812 06:55:31.648097 28945 raft_consensus.cc:2454] T ab96689a1eb640efac4c74fe5d8dfca8 P a2cecf39b4004e88aadc9378c23aa474 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b2b871392e9945ef93d6708b940f10e6 in term 1.
I20260812 06:55:31.648437 28709 leader_election.cc:302] T ab96689a1eb640efac4c74fe5d8dfca8 P b2b871392e9945ef93d6708b940f10e6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2cecf39b4004e88aadc9378c23aa474, b2b871392e9945ef93d6708b940f10e6; no voters:
I20260812 06:55:31.648677 29042 raft_consensus.cc:2781] T ab96689a1eb640efac4c74fe5d8dfca8 P b2b871392e9945ef93d6708b940f10e6 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 06:55:31.648795 29042 raft_consensus.cc:686] T ab96689a1eb640efac4c74fe5d8dfca8 P b2b871392e9945ef93d6708b940f10e6 [term 1 LEADER]: Becoming Leader. State: Replica: b2b871392e9945ef93d6708b940f10e6, State: Running, Role: LEADER
I20260812 06:55:31.648934 29042 consensus_queue.cc:237] T ab96689a1eb640efac4c74fe5d8dfca8 P b2b871392e9945ef93d6708b940f10e6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2cebfda268094256bb864f37e363b58c" member_type: VOTER last_known_addr { host: "127.19.52.65" port: 34495 } } peers { permanent_uuid: "b2b871392e9945ef93d6708b940f10e6" member_type: VOTER last_known_addr { host: "127.19.52.66" port: 38373 } } peers { permanent_uuid: "a2cecf39b4004e88aadc9378c23aa474" member_type: VOTER last_known_addr { host: "127.19.52.67" port: 44615 } }
I20260812 06:55:31.649669 29045 tablet_replica.cc:439] T ab96689a1eb640efac4c74fe5d8dfca8 P b2b871392e9945ef93d6708b940f10e6: TxnStatusTablet state changed. Reason: New leader b2b871392e9945ef93d6708b940f10e6. Latest consensus state: current_term: 1 leader_uuid: "b2b871392e9945ef93d6708b940f10e6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2cebfda268094256bb864f37e363b58c" member_type: VOTER last_known_addr { host: "127.19.52.65" port: 34495 } } peers { permanent_uuid: "b2b871392e9945ef93d6708b940f10e6" member_type: VOTER last_known_addr { host: "127.19.52.66" port: 38373 } } peers { permanent_uuid: "a2cecf39b4004e88aadc9378c23aa474" member_type: VOTER last_known_addr { host: "127.19.52.67" port: 44615 } } }
I20260812 06:55:31.649791 29045 tablet_replica.cc:442] T ab96689a1eb640efac4c74fe5d8dfca8 P b2b871392e9945ef93d6708b940f10e6: This TxnStatusTablet replica's current role is: LEADER
I20260812 06:55:31.650015 29083 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260812 06:55:31.650175 29083 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260812 06:55:31.651002 28448 catalog_manager.cc:5495] T ab96689a1eb640efac4c74fe5d8dfca8 P b2b871392e9945ef93d6708b940f10e6 reported cstate change: term changed from 0 to 1, leader changed from <none> to b2b871392e9945ef93d6708b940f10e6 (127.19.52.66). New cstate: current_term: 1 leader_uuid: "b2b871392e9945ef93d6708b940f10e6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2cebfda268094256bb864f37e363b58c" member_type: VOTER last_known_addr { host: "127.19.52.65" port: 34495 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b2b871392e9945ef93d6708b940f10e6" member_type: VOTER last_known_addr { host: "127.19.52.66" port: 38373 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a2cecf39b4004e88aadc9378c23aa474" member_type: VOTER last_known_addr { host: "127.19.52.67" port: 44615 } health_report { overall_health: UNKNOWN } } }
I20260812 06:55:31.675436 29029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:31.671556 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58894 (local address 127.19.52.66:38373)
0812 06:55:31.671686 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0812 06:55:31.671689 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 06:55:31.671785 (+ 96us) server_negotiation.cc:407] Connection header received
0812 06:55:31.672004 (+ 219us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:55:31.672006 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:55:31.672053 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:55:31.672122 (+ 69us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 06:55:31.672394 (+ 272us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:31.672957 (+ 563us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:31.674203 (+ 1246us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:31.674449 (+ 246us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:31.674535 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:55:31.674898 (+ 363us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 06:55:31.675026 (+ 128us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 06:55:31.675099 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0812 06:55:31.675255 (+ 156us) server_negotiation.cc:299] Negotiation successful
0812 06:55:31.675307 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260812 06:55:31.675453 29077 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:31.671435 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.66:38373 (local address 127.0.0.1:58894)
0812 06:55:31.671707 (+ 272us) negotiation.cc:107] Waiting for socket to connect
0812 06:55:31.671722 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 06:55:31.671837 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:55:31.672181 (+ 344us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:55:31.672184 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:55:31.672192 (+ 8us) client_negotiation.cc:189] Negotiated authn=TOKEN
0812 06:55:31.672307 (+ 115us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:55:31.672313 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:55:31.673069 (+ 756us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:55:31.673073 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:55:31.674074 (+ 1001us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:55:31.674085 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:55:31.674569 (+ 484us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:55:31.674573 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:55:31.674670 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:55:31.674730 (+ 60us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0812 06:55:31.675130 (+ 400us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0812 06:55:31.675146 (+ 16us) client_negotiation.cc:769] Sending connection context
0812 06:55:31.675216 (+ 70us) client_negotiation.cc:240] Negotiation successful
0812 06:55:31.675305 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":162}
I20260812 06:55:31.678385 28945 raft_consensus.cc:1261] T ab96689a1eb640efac4c74fe5d8dfca8 P a2cecf39b4004e88aadc9378c23aa474 [term 1 FOLLOWER]: Refusing update from remote peer b2b871392e9945ef93d6708b940f10e6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 06:55:31.678507 28628 raft_consensus.cc:1261] T ab96689a1eb640efac4c74fe5d8dfca8 P 2cebfda268094256bb864f37e363b58c [term 1 FOLLOWER]: Refusing update from remote peer b2b871392e9945ef93d6708b940f10e6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 06:55:31.678994 29045 consensus_queue.cc:1035] T ab96689a1eb640efac4c74fe5d8dfca8 P b2b871392e9945ef93d6708b940f10e6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2cecf39b4004e88aadc9378c23aa474" member_type: VOTER last_known_addr { host: "127.19.52.67" port: 44615 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 06:55:31.679198 29042 consensus_queue.cc:1035] T ab96689a1eb640efac4c74fe5d8dfca8 P b2b871392e9945ef93d6708b940f10e6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2cebfda268094256bb864f37e363b58c" member_type: VOTER last_known_addr { host: "127.19.52.65" port: 34495 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 06:55:31.681023 29041 tablet_replica.cc:439] T ab96689a1eb640efac4c74fe5d8dfca8 P a2cecf39b4004e88aadc9378c23aa474: TxnStatusTablet state changed. Reason: New leader b2b871392e9945ef93d6708b940f10e6. Latest consensus state: current_term: 1 leader_uuid: "b2b871392e9945ef93d6708b940f10e6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2cebfda268094256bb864f37e363b58c" member_type: VOTER last_known_addr { host: "127.19.52.65" port: 34495 } } peers { permanent_uuid: "b2b871392e9945ef93d6708b940f10e6" member_type: VOTER last_known_addr { host: "127.19.52.66" port: 38373 } } peers { permanent_uuid: "a2cecf39b4004e88aadc9378c23aa474" member_type: VOTER last_known_addr { host: "127.19.52.67" port: 44615 } } }
I20260812 06:55:31.681143 29041 tablet_replica.cc:442] T ab96689a1eb640efac4c74fe5d8dfca8 P a2cecf39b4004e88aadc9378c23aa474: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 06:55:31.681438 29039 tablet_replica.cc:439] T ab96689a1eb640efac4c74fe5d8dfca8 P 2cebfda268094256bb864f37e363b58c: TxnStatusTablet state changed. Reason: New leader b2b871392e9945ef93d6708b940f10e6. Latest consensus state: current_term: 1 leader_uuid: "b2b871392e9945ef93d6708b940f10e6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2cebfda268094256bb864f37e363b58c" member_type: VOTER last_known_addr { host: "127.19.52.65" port: 34495 } } peers { permanent_uuid: "b2b871392e9945ef93d6708b940f10e6" member_type: VOTER last_known_addr { host: "127.19.52.66" port: 38373 } } peers { permanent_uuid: "a2cecf39b4004e88aadc9378c23aa474" member_type: VOTER last_known_addr { host: "127.19.52.67" port: 44615 } } }
I20260812 06:55:31.681537 29039 tablet_replica.cc:442] T ab96689a1eb640efac4c74fe5d8dfca8 P 2cebfda268094256bb864f37e363b58c: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 06:55:31.682832 29042 tablet_replica.cc:439] T ab96689a1eb640efac4c74fe5d8dfca8 P b2b871392e9945ef93d6708b940f10e6: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "b2b871392e9945ef93d6708b940f10e6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2cebfda268094256bb864f37e363b58c" member_type: VOTER last_known_addr { host: "127.19.52.65" port: 34495 } } peers { permanent_uuid: "b2b871392e9945ef93d6708b940f10e6" member_type: VOTER last_known_addr { host: "127.19.52.66" port: 38373 } } peers { permanent_uuid: "a2cecf39b4004e88aadc9378c23aa474" member_type: VOTER last_known_addr { host: "127.19.52.67" port: 44615 } } }
I20260812 06:55:31.682955 29042 tablet_replica.cc:442] T ab96689a1eb640efac4c74fe5d8dfca8 P b2b871392e9945ef93d6708b940f10e6: This TxnStatusTablet replica's current role is: LEADER
I20260812 06:55:31.684321 29042 tablet_replica.cc:439] T ab96689a1eb640efac4c74fe5d8dfca8 P b2b871392e9945ef93d6708b940f10e6: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "b2b871392e9945ef93d6708b940f10e6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2cebfda268094256bb864f37e363b58c" member_type: VOTER last_known_addr { host: "127.19.52.65" port: 34495 } } peers { permanent_uuid: "b2b871392e9945ef93d6708b940f10e6" member_type: VOTER last_known_addr { host: "127.19.52.66" port: 38373 } } peers { permanent_uuid: "a2cecf39b4004e88aadc9378c23aa474" member_type: VOTER last_known_addr { host: "127.19.52.67" port: 44615 } } }
I20260812 06:55:31.684461 29042 tablet_replica.cc:442] T ab96689a1eb640efac4c74fe5d8dfca8 P b2b871392e9945ef93d6708b940f10e6: This TxnStatusTablet replica's current role is: LEADER
I20260812 06:55:31.684993 29041 tablet_replica.cc:439] T ab96689a1eb640efac4c74fe5d8dfca8 P a2cecf39b4004e88aadc9378c23aa474: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "b2b871392e9945ef93d6708b940f10e6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2cebfda268094256bb864f37e363b58c" member_type: VOTER last_known_addr { host: "127.19.52.65" port: 34495 } } peers { permanent_uuid: "b2b871392e9945ef93d6708b940f10e6" member_type: VOTER last_known_addr { host: "127.19.52.66" port: 38373 } } peers { permanent_uuid: "a2cecf39b4004e88aadc9378c23aa474" member_type: VOTER last_known_addr { host: "127.19.52.67" port: 44615 } } }
I20260812 06:55:31.685103 29041 tablet_replica.cc:442] T ab96689a1eb640efac4c74fe5d8dfca8 P a2cecf39b4004e88aadc9378c23aa474: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 06:55:31.686223 29039 tablet_replica.cc:439] T ab96689a1eb640efac4c74fe5d8dfca8 P 2cebfda268094256bb864f37e363b58c: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "b2b871392e9945ef93d6708b940f10e6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2cebfda268094256bb864f37e363b58c" member_type: VOTER last_known_addr { host: "127.19.52.65" port: 34495 } } peers { permanent_uuid: "b2b871392e9945ef93d6708b940f10e6" member_type: VOTER last_known_addr { host: "127.19.52.66" port: 38373 } } peers { permanent_uuid: "a2cecf39b4004e88aadc9378c23aa474" member_type: VOTER last_known_addr { host: "127.19.52.67" port: 44615 } } }
I20260812 06:55:31.686338 29039 tablet_replica.cc:442] T ab96689a1eb640efac4c74fe5d8dfca8 P 2cebfda268094256bb864f37e363b58c: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 06:55:31.694044 29030 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:31.689618 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59074 (local address 127.19.52.67:44615)
0812 06:55:31.689755 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0812 06:55:31.689759 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 06:55:31.689773 (+ 14us) server_negotiation.cc:407] Connection header received
0812 06:55:31.689922 (+ 149us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:55:31.689925 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:55:31.690002 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:55:31.690131 (+ 129us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 06:55:31.690422 (+ 291us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:31.691103 (+ 681us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:31.692035 (+ 932us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:31.692300 (+ 265us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:31.692427 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:55:31.692773 (+ 346us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 06:55:31.693371 (+ 598us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 06:55:31.693500 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0812 06:55:31.693700 (+ 200us) server_negotiation.cc:299] Negotiation successful
0812 06:55:31.693796 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260812 06:55:31.705575 29029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:31.701744 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.52.67:55213 (local address 127.19.52.66:38373)
0812 06:55:31.701900 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0812 06:55:31.701903 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 06:55:31.702025 (+ 122us) server_negotiation.cc:407] Connection header received
0812 06:55:31.702150 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:55:31.702153 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:55:31.702199 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:55:31.702263 (+ 64us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 06:55:31.702561 (+ 298us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:31.703081 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:31.704315 (+ 1234us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:31.704616 (+ 301us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:31.704709 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:55:31.705064 (+ 355us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 06:55:31.705151 (+ 87us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 06:55:31.705216 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0812 06:55:31.705383 (+ 167us) server_negotiation.cc:299] Negotiation successful
0812 06:55:31.705467 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260812 06:55:31.705583 29092 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:31.701568 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.66:38373 (local address 127.19.52.67:55213)
0812 06:55:31.701926 (+ 358us) negotiation.cc:107] Waiting for socket to connect
0812 06:55:31.701941 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 06:55:31.702048 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:55:31.702297 (+ 249us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:55:31.702301 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:55:31.702308 (+ 7us) client_negotiation.cc:189] Negotiated authn=TOKEN
0812 06:55:31.702433 (+ 125us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:55:31.702439 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:55:31.703191 (+ 752us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:55:31.703194 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:55:31.704167 (+ 973us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:55:31.704178 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:55:31.704731 (+ 553us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:55:31.704735 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:55:31.704836 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:55:31.704905 (+ 69us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0812 06:55:31.705244 (+ 339us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0812 06:55:31.705250 (+ 6us) client_negotiation.cc:769] Sending connection context
0812 06:55:31.705317 (+ 67us) client_negotiation.cc:240] Negotiation successful
0812 06:55:31.705395 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":256,"thread_start_us":159,"threads_started":1}
I20260812 06:55:31.713276 28628 raft_consensus.cc:1261] T 4183186e9bd24a8a861a59b01884aa93 P 2cebfda268094256bb864f37e363b58c [term 1 FOLLOWER]: Refusing update from remote peer a2cecf39b4004e88aadc9378c23aa474: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 06:55:31.713474 28788 raft_consensus.cc:1261] T 4183186e9bd24a8a861a59b01884aa93 P b2b871392e9945ef93d6708b940f10e6 [term 1 FOLLOWER]: Refusing update from remote peer a2cecf39b4004e88aadc9378c23aa474: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 06:55:31.714125 29069 consensus_queue.cc:1035] T 4183186e9bd24a8a861a59b01884aa93 P a2cecf39b4004e88aadc9378c23aa474 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2cebfda268094256bb864f37e363b58c" member_type: VOTER last_known_addr { host: "127.19.52.65" port: 34495 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 06:55:31.714351 29041 consensus_queue.cc:1035] T 4183186e9bd24a8a861a59b01884aa93 P a2cecf39b4004e88aadc9378c23aa474 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b2b871392e9945ef93d6708b940f10e6" member_type: VOTER last_known_addr { host: "127.19.52.66" port: 38373 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 06:55:31.717880 29051 mvcc.cc:205] Tried to move back new op lower bound from 7317576629092864000 to 7317576627396685824. Current Snapshot: MvccSnapshot[applied={T|T < 7317576629092864000}]
I20260812 06:55:31.740078 29030 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:31.735980 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.52.66:46373 (local address 127.19.52.67:44615)
0812 06:55:31.736100 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0812 06:55:31.736105 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 06:55:31.736146 (+ 41us) server_negotiation.cc:407] Connection header received
0812 06:55:31.736311 (+ 165us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:55:31.736314 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:55:31.736366 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:55:31.736453 (+ 87us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 06:55:31.736766 (+ 313us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:31.737435 (+ 669us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:31.738687 (+ 1252us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:31.739017 (+ 330us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:31.739140 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:55:31.739582 (+ 442us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 06:55:31.739673 (+ 91us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 06:55:31.739742 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0812 06:55:31.739885 (+ 143us) server_negotiation.cc:299] Negotiation successful
0812 06:55:31.739939 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260812 06:55:31.740459 29097 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:31.735623 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.67:44615 (local address 127.19.52.66:46373)
0812 06:55:31.736067 (+ 444us) negotiation.cc:107] Waiting for socket to connect
0812 06:55:31.736082 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 06:55:31.736189 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:55:31.736474 (+ 285us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:55:31.736477 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:55:31.736484 (+ 7us) client_negotiation.cc:189] Negotiated authn=TOKEN
0812 06:55:31.736639 (+ 155us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:55:31.736646 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:55:31.737545 (+ 899us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:55:31.737549 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:55:31.738546 (+ 997us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:55:31.738557 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:55:31.739234 (+ 677us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:55:31.739238 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:55:31.739367 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:55:31.739426 (+ 59us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0812 06:55:31.739783 (+ 357us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0812 06:55:31.739790 (+ 7us) client_negotiation.cc:769] Sending connection context
0812 06:55:31.740230 (+ 440us) client_negotiation.cc:240] Negotiation successful
0812 06:55:31.740317 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":264,"thread_start_us":131,"threads_started":1}
I20260812 06:55:31.766471 28447 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:42096:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 06:55:31.766750 28447 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:42096:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 06:55:31.769212 28447 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 3bce2c30f41c4196af2accc6509b44ff: Sending DeleteTablet for 3 replicas of tablet 4183186e9bd24a8a861a59b01884aa93
I20260812 06:55:31.770114 28607 tablet_service.cc:1514] Processing DeleteTablet for tablet 4183186e9bd24a8a861a59b01884aa93 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 06:55:31 UTC) from {username='oryx', principal='oryx/127.19.52.126@KRBTEST.COM'} at 127.0.0.1:53332
I20260812 06:55:31.770140 28764 tablet_service.cc:1514] Processing DeleteTablet for tablet 4183186e9bd24a8a861a59b01884aa93 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 06:55:31 UTC) from {username='oryx', principal='oryx/127.19.52.126@KRBTEST.COM'} at 127.0.0.1:58876
I20260812 06:55:31.770330 29062 tablet_replica.cc:330] T 4183186e9bd24a8a861a59b01884aa93 P b2b871392e9945ef93d6708b940f10e6: stopping tablet replica
I20260812 06:55:31.770351 29060 tablet_replica.cc:330] T 4183186e9bd24a8a861a59b01884aa93 P 2cebfda268094256bb864f37e363b58c: stopping tablet replica
I20260812 06:55:31.770433 29062 raft_consensus.cc:2229] T 4183186e9bd24a8a861a59b01884aa93 P b2b871392e9945ef93d6708b940f10e6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 06:55:31.770496 29060 raft_consensus.cc:2229] T 4183186e9bd24a8a861a59b01884aa93 P 2cebfda268094256bb864f37e363b58c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 06:55:31.770532 29062 raft_consensus.cc:2258] T 4183186e9bd24a8a861a59b01884aa93 P b2b871392e9945ef93d6708b940f10e6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 06:55:31.770640 29060 raft_consensus.cc:2258] T 4183186e9bd24a8a861a59b01884aa93 P 2cebfda268094256bb864f37e363b58c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 06:55:31.771440 29060 ts_tablet_manager.cc:1905] T 4183186e9bd24a8a861a59b01884aa93 P 2cebfda268094256bb864f37e363b58c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 06:55:31.771440 29062 ts_tablet_manager.cc:1905] T 4183186e9bd24a8a861a59b01884aa93 P b2b871392e9945ef93d6708b940f10e6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 06:55:31.773161 28922 tablet_service.cc:1514] Processing DeleteTablet for tablet 4183186e9bd24a8a861a59b01884aa93 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 06:55:31 UTC) from {username='oryx', principal='oryx/127.19.52.126@KRBTEST.COM'} at 127.0.0.1:59064
I20260812 06:55:31.774780 29060 ts_tablet_manager.cc:1918] T 4183186e9bd24a8a861a59b01884aa93 P 2cebfda268094256bb864f37e363b58c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260812 06:55:31.774878 29060 log.cc:1196] T 4183186e9bd24a8a861a59b01884aa93 P 2cebfda268094256bb864f37e363b58c: Deleting WAL directory at /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786517687384214-19665-0/minicluster-data/ts-0/wal/wals/4183186e9bd24a8a861a59b01884aa93
I20260812 06:55:31.775159 29060 ts_tablet_manager.cc:1939] T 4183186e9bd24a8a861a59b01884aa93 P 2cebfda268094256bb864f37e363b58c: Deleting consensus metadata
I20260812 06:55:31.779104 28433 catalog_manager.cc:4841] TS 2cebfda268094256bb864f37e363b58c (127.19.52.65:34495): tablet 4183186e9bd24a8a861a59b01884aa93 (table test-table [id=f8bc3c9400f8435ab1dc221b8cf49dd2]) successfully deleted
I20260812 06:55:31.781740 29062 ts_tablet_manager.cc:1918] T 4183186e9bd24a8a861a59b01884aa93 P b2b871392e9945ef93d6708b940f10e6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260812 06:55:31.781852 29062 log.cc:1196] T 4183186e9bd24a8a861a59b01884aa93 P b2b871392e9945ef93d6708b940f10e6: Deleting WAL directory at /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786517687384214-19665-0/minicluster-data/ts-1/wal/wals/4183186e9bd24a8a861a59b01884aa93
I20260812 06:55:31.782141 29062 ts_tablet_manager.cc:1939] T 4183186e9bd24a8a861a59b01884aa93 P b2b871392e9945ef93d6708b940f10e6: Deleting consensus metadata
I20260812 06:55:31.783042 28433 catalog_manager.cc:4841] TS b2b871392e9945ef93d6708b940f10e6 (127.19.52.66:38373): tablet 4183186e9bd24a8a861a59b01884aa93 (table test-table [id=f8bc3c9400f8435ab1dc221b8cf49dd2]) successfully deleted
I20260812 06:55:31.784322 29100 tablet_replica.cc:330] T 4183186e9bd24a8a861a59b01884aa93 P a2cecf39b4004e88aadc9378c23aa474: stopping tablet replica
I20260812 06:55:31.784605 29100 raft_consensus.cc:2229] T 4183186e9bd24a8a861a59b01884aa93 P a2cecf39b4004e88aadc9378c23aa474 [term 1 LEADER]: Raft consensus shutting down.
I20260812 06:55:31.785040 29100 raft_consensus.cc:2258] T 4183186e9bd24a8a861a59b01884aa93 P a2cecf39b4004e88aadc9378c23aa474 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 06:55:31.785867 29100 ts_tablet_manager.cc:1905] T 4183186e9bd24a8a861a59b01884aa93 P a2cecf39b4004e88aadc9378c23aa474: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 06:55:31.789873 29100 ts_tablet_manager.cc:1918] T 4183186e9bd24a8a861a59b01884aa93 P a2cecf39b4004e88aadc9378c23aa474: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260812 06:55:31.790030 29100 log.cc:1196] T 4183186e9bd24a8a861a59b01884aa93 P a2cecf39b4004e88aadc9378c23aa474: Deleting WAL directory at /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786517687384214-19665-0/minicluster-data/ts-2/wal/wals/4183186e9bd24a8a861a59b01884aa93
I20260812 06:55:31.790392 29100 ts_tablet_manager.cc:1939] T 4183186e9bd24a8a861a59b01884aa93 P a2cecf39b4004e88aadc9378c23aa474: Deleting consensus metadata
I20260812 06:55:31.791483 28434 catalog_manager.cc:4841] TS a2cecf39b4004e88aadc9378c23aa474 (127.19.52.67:44615): tablet 4183186e9bd24a8a861a59b01884aa93 (table test-table [id=f8bc3c9400f8435ab1dc221b8cf49dd2]) successfully deleted
Aug 12 06:55:31 dist-test-slave-65mx krb5kdc[28399](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517731, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 12 06:55:31 dist-test-slave-65mx krb5kdc[28399](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.19.52.126@KRBTEST.COM, Server not found in Kerberos database
Aug 12 06:55:31 dist-test-slave-65mx krb5kdc[28399](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.19.52.126@KRBTEST.COM, Server not found in Kerberos database
W20260812 06:55:31.807569 28504 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 06:55:31.800480 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42114 (local address 127.19.52.126:41321)
0812 06:55:31.800702 (+ 222us) server_negotiation.cc:206] Beginning negotiation
0812 06:55:31.800707 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 06:55:31.800973 (+ 266us) server_negotiation.cc:407] Connection header received
0812 06:55:31.801149 (+ 176us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:55:31.801153 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:55:31.801213 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:55:31.801308 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:55:31.802257 (+ 949us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:31.802807 (+ 550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:31.803715 (+ 908us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:31.803959 (+ 244us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:31.804076 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:55:31.807461 (+ 3385us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42114: BlockingRecv error: recv got EOF from 127.0.0.1:42114 (error 108)
Metrics: {"server-negotiator.queue_time_us":129}
Aug 12 06:55:31 dist-test-slave-65mx krb5kdc[28399](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517731, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.19.52.126@KRBTEST.COM
I20260812 06:55:31.819511 28504 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:31.810178 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42116 (local address 127.19.52.126:41321)
0812 06:55:31.810717 (+ 539us) server_negotiation.cc:206] Beginning negotiation
0812 06:55:31.810720 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 06:55:31.810734 (+ 14us) server_negotiation.cc:407] Connection header received
0812 06:55:31.810771 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:55:31.810773 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:55:31.810813 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:55:31.810915 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:55:31.811573 (+ 658us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:31.812092 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:31.812957 (+ 865us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:31.813181 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:31.813297 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:55:31.816051 (+ 2754us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:55:31.816055 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:55:31.816057 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:55:31.816083 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:55:31.817786 (+ 1703us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:55:31.818230 (+ 444us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:55:31.818233 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:55:31.818235 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:55:31.818321 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:55:31.818616 (+ 295us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:55:31.818619 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:55:31.818621 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:55:31.818815 (+ 194us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:55:31.818960 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0812 06:55:31.819197 (+ 237us) server_negotiation.cc:299] Negotiation successful
0812 06:55:31.819318 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":463}
I20260812 06:55:31.822438 28447 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42116:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 06:55:31.822944 28447 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 06:55:31.831728 28764 tablet_service.cc:1467] Processing CreateTablet for tablet da471364193c45e2a1245923620ff1f2 (DEFAULT_TABLE table=test-table [id=ad67abaa469142769bd438b2beca8fa0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 06:55:31.831728 28922 tablet_service.cc:1467] Processing CreateTablet for tablet da471364193c45e2a1245923620ff1f2 (DEFAULT_TABLE table=test-table [id=ad67abaa469142769bd438b2beca8fa0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 06:55:31.832074 28764 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet da471364193c45e2a1245923620ff1f2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 06:55:31.832074 28922 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet da471364193c45e2a1245923620ff1f2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 06:55:31.834858 28607 tablet_service.cc:1467] Processing CreateTablet for tablet da471364193c45e2a1245923620ff1f2 (DEFAULT_TABLE table=test-table [id=ad67abaa469142769bd438b2beca8fa0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 06:55:31.835184 28607 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet da471364193c45e2a1245923620ff1f2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 06:55:31.836897 29032 tablet_bootstrap.cc:492] T da471364193c45e2a1245923620ff1f2 P a2cecf39b4004e88aadc9378c23aa474: Bootstrap starting.
I20260812 06:55:31.837361 29031 tablet_bootstrap.cc:492] T da471364193c45e2a1245923620ff1f2 P 2cebfda268094256bb864f37e363b58c: Bootstrap starting.
I20260812 06:55:31.837925 29032 tablet_bootstrap.cc:654] T da471364193c45e2a1245923620ff1f2 P a2cecf39b4004e88aadc9378c23aa474: Neither blocks nor log segments found. Creating new log.
I20260812 06:55:31.838096 29031 tablet_bootstrap.cc:654] T da471364193c45e2a1245923620ff1f2 P 2cebfda268094256bb864f37e363b58c: Neither blocks nor log segments found. Creating new log.
I20260812 06:55:31.838586 29033 tablet_bootstrap.cc:492] T da471364193c45e2a1245923620ff1f2 P b2b871392e9945ef93d6708b940f10e6: Bootstrap starting.
I20260812 06:55:31.839257 29032 tablet_bootstrap.cc:492] T da471364193c45e2a1245923620ff1f2 P a2cecf39b4004e88aadc9378c23aa474: No bootstrap required, opened a new log
I20260812 06:55:31.839346 29032 ts_tablet_manager.cc:1397] T da471364193c45e2a1245923620ff1f2 P a2cecf39b4004e88aadc9378c23aa474: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260812 06:55:31.839403 29033 tablet_bootstrap.cc:654] T da471364193c45e2a1245923620ff1f2 P b2b871392e9945ef93d6708b940f10e6: Neither blocks nor log segments found. Creating new log.
I20260812 06:55:31.839525 29031 tablet_bootstrap.cc:492] T da471364193c45e2a1245923620ff1f2 P 2cebfda268094256bb864f37e363b58c: No bootstrap required, opened a new log
I20260812 06:55:31.839596 29031 ts_tablet_manager.cc:1397] T da471364193c45e2a1245923620ff1f2 P 2cebfda268094256bb864f37e363b58c: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260812 06:55:31.839831 29032 raft_consensus.cc:348] T da471364193c45e2a1245923620ff1f2 P a2cecf39b4004e88aadc9378c23aa474 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2cebfda268094256bb864f37e363b58c" member_type: VOTER last_known_addr { host: "127.19.52.65" port: 34495 } } peers { permanent_uuid: "b2b871392e9945ef93d6708b940f10e6" member_type: VOTER last_known_addr { host: "127.19.52.66" port: 38373 } } peers { permanent_uuid: "a2cecf39b4004e88aadc9378c23aa474" member_type: VOTER last_known_addr { host: "127.19.52.67" port: 44615 } }
I20260812 06:55:31.839953 29032 raft_consensus.cc:374] T da471364193c45e2a1245923620ff1f2 P a2cecf39b4004e88aadc9378c23aa474 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 06:55:31.839993 29032 raft_consensus.cc:729] T da471364193c45e2a1245923620ff1f2 P a2cecf39b4004e88aadc9378c23aa474 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a2cecf39b4004e88aadc9378c23aa474, State: Initialized, Role: FOLLOWER
I20260812 06:55:31.839993 29031 raft_consensus.cc:348] T da471364193c45e2a1245923620ff1f2 P 2cebfda268094256bb864f37e363b58c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2cebfda268094256bb864f37e363b58c" member_type: VOTER last_known_addr { host: "127.19.52.65" port: 34495 } } peers { permanent_uuid: "b2b871392e9945ef93d6708b940f10e6" member_type: VOTER last_known_addr { host: "127.19.52.66" port: 38373 } } peers { permanent_uuid: "a2cecf39b4004e88aadc9378c23aa474" member_type: VOTER last_known_addr { host: "127.19.52.67" port: 44615 } }
I20260812 06:55:31.840095 29031 raft_consensus.cc:374] T da471364193c45e2a1245923620ff1f2 P 2cebfda268094256bb864f37e363b58c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 06:55:31.840126 29031 raft_consensus.cc:729] T da471364193c45e2a1245923620ff1f2 P 2cebfda268094256bb864f37e363b58c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2cebfda268094256bb864f37e363b58c, State: Initialized, Role: FOLLOWER
I20260812 06:55:31.840134 29032 consensus_queue.cc:260] T da471364193c45e2a1245923620ff1f2 P a2cecf39b4004e88aadc9378c23aa474 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2cebfda268094256bb864f37e363b58c" member_type: VOTER last_known_addr { host: "127.19.52.65" port: 34495 } } peers { permanent_uuid: "b2b871392e9945ef93d6708b940f10e6" member_type: VOTER last_known_addr { host: "127.19.52.66" port: 38373 } } peers { permanent_uuid: "a2cecf39b4004e88aadc9378c23aa474" member_type: VOTER last_known_addr { host: "127.19.52.67" port: 44615 } }
I20260812 06:55:31.840240 29031 consensus_queue.cc:260] T da471364193c45e2a1245923620ff1f2 P 2cebfda268094256bb864f37e363b58c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2cebfda268094256bb864f37e363b58c" member_type: VOTER last_known_addr { host: "127.19.52.65" port: 34495 } } peers { permanent_uuid: "b2b871392e9945ef93d6708b940f10e6" member_type: VOTER last_known_addr { host: "127.19.52.66" port: 38373 } } peers { permanent_uuid: "a2cecf39b4004e88aadc9378c23aa474" member_type: VOTER last_known_addr { host: "127.19.52.67" port: 44615 } }
I20260812 06:55:31.840413 29032 ts_tablet_manager.cc:1428] T da471364193c45e2a1245923620ff1f2 P a2cecf39b4004e88aadc9378c23aa474: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260812 06:55:31.840495 29031 ts_tablet_manager.cc:1428] T da471364193c45e2a1245923620ff1f2 P 2cebfda268094256bb864f37e363b58c: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260812 06:55:31.844110 29033 tablet_bootstrap.cc:492] T da471364193c45e2a1245923620ff1f2 P b2b871392e9945ef93d6708b940f10e6: No bootstrap required, opened a new log
I20260812 06:55:31.844229 29033 ts_tablet_manager.cc:1397] T da471364193c45e2a1245923620ff1f2 P b2b871392e9945ef93d6708b940f10e6: Time spent bootstrapping tablet: real 0.006s user 0.001s sys 0.000s
I20260812 06:55:31.844682 29033 raft_consensus.cc:348] T da471364193c45e2a1245923620ff1f2 P b2b871392e9945ef93d6708b940f10e6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2cebfda268094256bb864f37e363b58c" member_type: VOTER last_known_addr { host: "127.19.52.65" port: 34495 } } peers { permanent_uuid: "b2b871392e9945ef93d6708b940f10e6" member_type: VOTER last_known_addr { host: "127.19.52.66" port: 38373 } } peers { permanent_uuid: "a2cecf39b4004e88aadc9378c23aa474" member_type: VOTER last_known_addr { host: "127.19.52.67" port: 44615 } }
I20260812 06:55:31.844805 29033 raft_consensus.cc:374] T da471364193c45e2a1245923620ff1f2 P b2b871392e9945ef93d6708b940f10e6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 06:55:31.844851 29033 raft_consensus.cc:729] T da471364193c45e2a1245923620ff1f2 P b2b871392e9945ef93d6708b940f10e6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b2b871392e9945ef93d6708b940f10e6, State: Initialized, Role: FOLLOWER
I20260812 06:55:31.845000 29033 consensus_queue.cc:260] T da471364193c45e2a1245923620ff1f2 P b2b871392e9945ef93d6708b940f10e6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2cebfda268094256bb864f37e363b58c" member_type: VOTER last_known_addr { host: "127.19.52.65" port: 34495 } } peers { permanent_uuid: "b2b871392e9945ef93d6708b940f10e6" member_type: VOTER last_known_addr { host: "127.19.52.66" port: 38373 } } peers { permanent_uuid: "a2cecf39b4004e88aadc9378c23aa474" member_type: VOTER last_known_addr { host: "127.19.52.67" port: 44615 } }
I20260812 06:55:31.845292 29033 ts_tablet_manager.cc:1428] T da471364193c45e2a1245923620ff1f2 P b2b871392e9945ef93d6708b940f10e6: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260812 06:55:31.941181 29095 raft_consensus.cc:482] T da471364193c45e2a1245923620ff1f2 P a2cecf39b4004e88aadc9378c23aa474 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 06:55:31.941342 29095 raft_consensus.cc:504] T da471364193c45e2a1245923620ff1f2 P a2cecf39b4004e88aadc9378c23aa474 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2cebfda268094256bb864f37e363b58c" member_type: VOTER last_known_addr { host: "127.19.52.65" port: 34495 } } peers { permanent_uuid: "b2b871392e9945ef93d6708b940f10e6" member_type: VOTER last_known_addr { host: "127.19.52.66" port: 38373 } } peers { permanent_uuid: "a2cecf39b4004e88aadc9378c23aa474" member_type: VOTER last_known_addr { host: "127.19.52.67" port: 44615 } }
I20260812 06:55:31.941841 29095 leader_election.cc:288] T da471364193c45e2a1245923620ff1f2 P a2cecf39b4004e88aadc9378c23aa474 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2cebfda268094256bb864f37e363b58c (127.19.52.65:34495), b2b871392e9945ef93d6708b940f10e6 (127.19.52.66:38373)
I20260812 06:55:31.942299 28628 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "da471364193c45e2a1245923620ff1f2" candidate_uuid: "a2cecf39b4004e88aadc9378c23aa474" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2cebfda268094256bb864f37e363b58c" is_pre_election: true
I20260812 06:55:31.942363 28788 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "da471364193c45e2a1245923620ff1f2" candidate_uuid: "a2cecf39b4004e88aadc9378c23aa474" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b2b871392e9945ef93d6708b940f10e6" is_pre_election: true
I20260812 06:55:31.942471 28628 raft_consensus.cc:2454] T da471364193c45e2a1245923620ff1f2 P 2cebfda268094256bb864f37e363b58c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2cecf39b4004e88aadc9378c23aa474 in term 0.
I20260812 06:55:31.942533 28788 raft_consensus.cc:2454] T da471364193c45e2a1245923620ff1f2 P b2b871392e9945ef93d6708b940f10e6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2cecf39b4004e88aadc9378c23aa474 in term 0.
I20260812 06:55:31.942832 28869 leader_election.cc:302] T da471364193c45e2a1245923620ff1f2 P a2cecf39b4004e88aadc9378c23aa474 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2cebfda268094256bb864f37e363b58c, a2cecf39b4004e88aadc9378c23aa474; no voters:
I20260812 06:55:31.943030 29095 raft_consensus.cc:2781] T da471364193c45e2a1245923620ff1f2 P a2cecf39b4004e88aadc9378c23aa474 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 06:55:31.943092 29095 raft_consensus.cc:482] T da471364193c45e2a1245923620ff1f2 P a2cecf39b4004e88aadc9378c23aa474 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 06:55:31.943135 29095 raft_consensus.cc:3037] T da471364193c45e2a1245923620ff1f2 P a2cecf39b4004e88aadc9378c23aa474 [term 0 FOLLOWER]: Advancing to term 1
I20260812 06:55:31.944108 29095 raft_consensus.cc:504] T da471364193c45e2a1245923620ff1f2 P a2cecf39b4004e88aadc9378c23aa474 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2cebfda268094256bb864f37e363b58c" member_type: VOTER last_known_addr { host: "127.19.52.65" port: 34495 } } peers { permanent_uuid: "b2b871392e9945ef93d6708b940f10e6" member_type: VOTER last_known_addr { host: "127.19.52.66" port: 38373 } } peers { permanent_uuid: "a2cecf39b4004e88aadc9378c23aa474" member_type: VOTER last_known_addr { host: "127.19.52.67" port: 44615 } }
I20260812 06:55:31.944478 29095 leader_election.cc:288] T da471364193c45e2a1245923620ff1f2 P a2cecf39b4004e88aadc9378c23aa474 [CANDIDATE]: Term 1 election: Requested vote from peers 2cebfda268094256bb864f37e363b58c (127.19.52.65:34495), b2b871392e9945ef93d6708b940f10e6 (127.19.52.66:38373)
I20260812 06:55:31.944846 28628 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "da471364193c45e2a1245923620ff1f2" candidate_uuid: "a2cecf39b4004e88aadc9378c23aa474" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2cebfda268094256bb864f37e363b58c"
I20260812 06:55:31.944896 28788 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "da471364193c45e2a1245923620ff1f2" candidate_uuid: "a2cecf39b4004e88aadc9378c23aa474" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b2b871392e9945ef93d6708b940f10e6"
I20260812 06:55:31.945010 28788 raft_consensus.cc:3037] T da471364193c45e2a1245923620ff1f2 P b2b871392e9945ef93d6708b940f10e6 [term 0 FOLLOWER]: Advancing to term 1
I20260812 06:55:31.945001 28628 raft_consensus.cc:3037] T da471364193c45e2a1245923620ff1f2 P 2cebfda268094256bb864f37e363b58c [term 0 FOLLOWER]: Advancing to term 1
I20260812 06:55:31.945901 28628 raft_consensus.cc:2454] T da471364193c45e2a1245923620ff1f2 P 2cebfda268094256bb864f37e363b58c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2cecf39b4004e88aadc9378c23aa474 in term 1.
I20260812 06:55:31.946199 28869 leader_election.cc:302] T da471364193c45e2a1245923620ff1f2 P a2cecf39b4004e88aadc9378c23aa474 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2cebfda268094256bb864f37e363b58c, a2cecf39b4004e88aadc9378c23aa474; no voters:
I20260812 06:55:31.946213 28788 raft_consensus.cc:2454] T da471364193c45e2a1245923620ff1f2 P b2b871392e9945ef93d6708b940f10e6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2cecf39b4004e88aadc9378c23aa474 in term 1.
I20260812 06:55:31.946403 29095 raft_consensus.cc:2781] T da471364193c45e2a1245923620ff1f2 P a2cecf39b4004e88aadc9378c23aa474 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 06:55:31.946496 29095 raft_consensus.cc:686] T da471364193c45e2a1245923620ff1f2 P a2cecf39b4004e88aadc9378c23aa474 [term 1 LEADER]: Becoming Leader. State: Replica: a2cecf39b4004e88aadc9378c23aa474, State: Running, Role: LEADER
I20260812 06:55:31.946659 29095 consensus_queue.cc:237] T da471364193c45e2a1245923620ff1f2 P a2cecf39b4004e88aadc9378c23aa474 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2cebfda268094256bb864f37e363b58c" member_type: VOTER last_known_addr { host: "127.19.52.65" port: 34495 } } peers { permanent_uuid: "b2b871392e9945ef93d6708b940f10e6" member_type: VOTER last_known_addr { host: "127.19.52.66" port: 38373 } } peers { permanent_uuid: "a2cecf39b4004e88aadc9378c23aa474" member_type: VOTER last_known_addr { host: "127.19.52.67" port: 44615 } }
I20260812 06:55:31.948589 28449 catalog_manager.cc:5495] T da471364193c45e2a1245923620ff1f2 P a2cecf39b4004e88aadc9378c23aa474 reported cstate change: term changed from 0 to 1, leader changed from <none> to a2cecf39b4004e88aadc9378c23aa474 (127.19.52.67). New cstate: current_term: 1 leader_uuid: "a2cecf39b4004e88aadc9378c23aa474" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2cebfda268094256bb864f37e363b58c" member_type: VOTER last_known_addr { host: "127.19.52.65" port: 34495 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b2b871392e9945ef93d6708b940f10e6" member_type: VOTER last_known_addr { host: "127.19.52.66" port: 38373 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a2cecf39b4004e88aadc9378c23aa474" member_type: VOTER last_known_addr { host: "127.19.52.67" port: 44615 } health_report { overall_health: HEALTHY } } }
I20260812 06:55:31.975572 29030 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:31.970143 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59086 (local address 127.19.52.67:44615)
0812 06:55:31.970340 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0812 06:55:31.970348 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 06:55:31.970382 (+ 34us) server_negotiation.cc:407] Connection header received
0812 06:55:31.970568 (+ 186us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:55:31.970571 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:55:31.970659 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:55:31.970767 (+ 108us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 06:55:31.971401 (+ 634us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:31.972207 (+ 806us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:31.973913 (+ 1706us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:31.974288 (+ 375us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:31.974428 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:55:31.974904 (+ 476us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 06:55:31.975063 (+ 159us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 06:55:31.975160 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0812 06:55:31.975327 (+ 167us) server_negotiation.cc:299] Negotiation successful
0812 06:55:31.975407 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260812 06:55:31.978983 28788 raft_consensus.cc:1261] T da471364193c45e2a1245923620ff1f2 P b2b871392e9945ef93d6708b940f10e6 [term 1 FOLLOWER]: Refusing update from remote peer a2cecf39b4004e88aadc9378c23aa474: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 06:55:31.979043 28628 raft_consensus.cc:1261] T da471364193c45e2a1245923620ff1f2 P 2cebfda268094256bb864f37e363b58c [term 1 FOLLOWER]: Refusing update from remote peer a2cecf39b4004e88aadc9378c23aa474: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 06:55:31.979759 29069 consensus_queue.cc:1035] T da471364193c45e2a1245923620ff1f2 P a2cecf39b4004e88aadc9378c23aa474 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b2b871392e9945ef93d6708b940f10e6" member_type: VOTER last_known_addr { host: "127.19.52.66" port: 38373 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 06:55:31.980026 29095 consensus_queue.cc:1035] T da471364193c45e2a1245923620ff1f2 P a2cecf39b4004e88aadc9378c23aa474 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2cebfda268094256bb864f37e363b58c" member_type: VOTER last_known_addr { host: "127.19.52.65" port: 34495 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 06:55:31.997581 28447 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42116:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 06:55:31.997881 28447 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42116:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 06:55:32.001197 28447 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 3bce2c30f41c4196af2accc6509b44ff: Sending DeleteTablet for 3 replicas of tablet da471364193c45e2a1245923620ff1f2
I20260812 06:55:32.002355 28607 tablet_service.cc:1514] Processing DeleteTablet for tablet da471364193c45e2a1245923620ff1f2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 06:55:31 UTC) from {username='oryx', principal='oryx/127.19.52.126@KRBTEST.COM'} at 127.0.0.1:53332
I20260812 06:55:32.002359 28764 tablet_service.cc:1514] Processing DeleteTablet for tablet da471364193c45e2a1245923620ff1f2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 06:55:31 UTC) from {username='oryx', principal='oryx/127.19.52.126@KRBTEST.COM'} at 127.0.0.1:58876
I20260812 06:55:32.002717 29060 tablet_replica.cc:330] T da471364193c45e2a1245923620ff1f2 P 2cebfda268094256bb864f37e363b58c: stopping tablet replica
I20260812 06:55:32.002803 29062 tablet_replica.cc:330] T da471364193c45e2a1245923620ff1f2 P b2b871392e9945ef93d6708b940f10e6: stopping tablet replica
I20260812 06:55:32.002781 28922 tablet_service.cc:1514] Processing DeleteTablet for tablet da471364193c45e2a1245923620ff1f2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 06:55:31 UTC) from {username='oryx', principal='oryx/127.19.52.126@KRBTEST.COM'} at 127.0.0.1:59064
I20260812 06:55:32.002946 29060 raft_consensus.cc:2229] T da471364193c45e2a1245923620ff1f2 P 2cebfda268094256bb864f37e363b58c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 06:55:32.003022 29062 raft_consensus.cc:2229] T da471364193c45e2a1245923620ff1f2 P b2b871392e9945ef93d6708b940f10e6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 06:55:32.003177 29060 raft_consensus.cc:2258] T da471364193c45e2a1245923620ff1f2 P 2cebfda268094256bb864f37e363b58c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 06:55:32.003238 29062 raft_consensus.cc:2258] T da471364193c45e2a1245923620ff1f2 P b2b871392e9945ef93d6708b940f10e6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 06:55:32.003298 29100 tablet_replica.cc:330] T da471364193c45e2a1245923620ff1f2 P a2cecf39b4004e88aadc9378c23aa474: stopping tablet replica
I20260812 06:55:32.003537 29100 raft_consensus.cc:2229] T da471364193c45e2a1245923620ff1f2 P a2cecf39b4004e88aadc9378c23aa474 [term 1 LEADER]: Raft consensus shutting down.
I20260812 06:55:32.004002 29100 raft_consensus.cc:2258] T da471364193c45e2a1245923620ff1f2 P a2cecf39b4004e88aadc9378c23aa474 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 06:55:32.004328 28447 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42116:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260812 06:55:32.004788 29060 ts_tablet_manager.cc:1905] T da471364193c45e2a1245923620ff1f2 P 2cebfda268094256bb864f37e363b58c: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260812 06:55:32.005183 28447 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 06:55:32.005198 29100 ts_tablet_manager.cc:1905] T da471364193c45e2a1245923620ff1f2 P a2cecf39b4004e88aadc9378c23aa474: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 06:55:32.007683 29062 ts_tablet_manager.cc:1905] T da471364193c45e2a1245923620ff1f2 P b2b871392e9945ef93d6708b940f10e6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 06:55:32.009392 29100 ts_tablet_manager.cc:1918] T da471364193c45e2a1245923620ff1f2 P a2cecf39b4004e88aadc9378c23aa474: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 06:55:32.009439 29060 ts_tablet_manager.cc:1918] T da471364193c45e2a1245923620ff1f2 P 2cebfda268094256bb864f37e363b58c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 06:55:32.009531 29100 log.cc:1196] T da471364193c45e2a1245923620ff1f2 P a2cecf39b4004e88aadc9378c23aa474: Deleting WAL directory at /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786517687384214-19665-0/minicluster-data/ts-2/wal/wals/da471364193c45e2a1245923620ff1f2
I20260812 06:55:32.009593 29060 log.cc:1196] T da471364193c45e2a1245923620ff1f2 P 2cebfda268094256bb864f37e363b58c: Deleting WAL directory at /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786517687384214-19665-0/minicluster-data/ts-0/wal/wals/da471364193c45e2a1245923620ff1f2
I20260812 06:55:32.010026 29100 ts_tablet_manager.cc:1939] T da471364193c45e2a1245923620ff1f2 P a2cecf39b4004e88aadc9378c23aa474: Deleting consensus metadata
I20260812 06:55:32.010191 29060 ts_tablet_manager.cc:1939] T da471364193c45e2a1245923620ff1f2 P 2cebfda268094256bb864f37e363b58c: Deleting consensus metadata
I20260812 06:55:32.011312 28434 catalog_manager.cc:4841] TS a2cecf39b4004e88aadc9378c23aa474 (127.19.52.67:44615): tablet da471364193c45e2a1245923620ff1f2 (table test-table [id=ad67abaa469142769bd438b2beca8fa0]) successfully deleted
I20260812 06:55:32.011904 29062 ts_tablet_manager.cc:1918] T da471364193c45e2a1245923620ff1f2 P b2b871392e9945ef93d6708b940f10e6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 06:55:32.012014 29062 log.cc:1196] T da471364193c45e2a1245923620ff1f2 P b2b871392e9945ef93d6708b940f10e6: Deleting WAL directory at /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786517687384214-19665-0/minicluster-data/ts-1/wal/wals/da471364193c45e2a1245923620ff1f2
I20260812 06:55:32.012181 28433 catalog_manager.cc:4841] TS 2cebfda268094256bb864f37e363b58c (127.19.52.65:34495): tablet da471364193c45e2a1245923620ff1f2 (table test-table [id=ad67abaa469142769bd438b2beca8fa0]) successfully deleted
I20260812 06:55:32.012535 29062 ts_tablet_manager.cc:1939] T da471364193c45e2a1245923620ff1f2 P b2b871392e9945ef93d6708b940f10e6: Deleting consensus metadata
I20260812 06:55:32.014650 28433 catalog_manager.cc:4841] TS b2b871392e9945ef93d6708b940f10e6 (127.19.52.66:38373): tablet da471364193c45e2a1245923620ff1f2 (table test-table [id=ad67abaa469142769bd438b2beca8fa0]) successfully deleted
I20260812 06:55:32.015336 28607 tablet_service.cc:1467] Processing CreateTablet for tablet 2cf091137a0f49599d022810fac45f0e (DEFAULT_TABLE table=test-table [id=9f0d9a94819a4f1aa6c1f633289ed7ae]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 06:55:32.015857 28607 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2cf091137a0f49599d022810fac45f0e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 06:55:32.015704 28764 tablet_service.cc:1467] Processing CreateTablet for tablet 2cf091137a0f49599d022810fac45f0e (DEFAULT_TABLE table=test-table [id=9f0d9a94819a4f1aa6c1f633289ed7ae]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 06:55:32.015796 28922 tablet_service.cc:1467] Processing CreateTablet for tablet 2cf091137a0f49599d022810fac45f0e (DEFAULT_TABLE table=test-table [id=9f0d9a94819a4f1aa6c1f633289ed7ae]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 06:55:32.016229 28922 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2cf091137a0f49599d022810fac45f0e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 06:55:32.016285 28764 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2cf091137a0f49599d022810fac45f0e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 06:55:32.019570 29033 tablet_bootstrap.cc:492] T 2cf091137a0f49599d022810fac45f0e P b2b871392e9945ef93d6708b940f10e6: Bootstrap starting.
I20260812 06:55:32.020260 29031 tablet_bootstrap.cc:492] T 2cf091137a0f49599d022810fac45f0e P 2cebfda268094256bb864f37e363b58c: Bootstrap starting.
I20260812 06:55:32.020792 29032 tablet_bootstrap.cc:492] T 2cf091137a0f49599d022810fac45f0e P a2cecf39b4004e88aadc9378c23aa474: Bootstrap starting.
I20260812 06:55:32.021643 29033 tablet_bootstrap.cc:654] T 2cf091137a0f49599d022810fac45f0e P b2b871392e9945ef93d6708b940f10e6: Neither blocks nor log segments found. Creating new log.
I20260812 06:55:32.021803 29032 tablet_bootstrap.cc:654] T 2cf091137a0f49599d022810fac45f0e P a2cecf39b4004e88aadc9378c23aa474: Neither blocks nor log segments found. Creating new log.
I20260812 06:55:32.022902 29031 tablet_bootstrap.cc:654] T 2cf091137a0f49599d022810fac45f0e P 2cebfda268094256bb864f37e363b58c: Neither blocks nor log segments found. Creating new log.
I20260812 06:55:32.024772 29033 tablet_bootstrap.cc:492] T 2cf091137a0f49599d022810fac45f0e P b2b871392e9945ef93d6708b940f10e6: No bootstrap required, opened a new log
I20260812 06:55:32.024930 29032 tablet_bootstrap.cc:492] T 2cf091137a0f49599d022810fac45f0e P a2cecf39b4004e88aadc9378c23aa474: No bootstrap required, opened a new log
I20260812 06:55:32.025043 29032 ts_tablet_manager.cc:1397] T 2cf091137a0f49599d022810fac45f0e P a2cecf39b4004e88aadc9378c23aa474: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260812 06:55:32.025032 29033 ts_tablet_manager.cc:1397] T 2cf091137a0f49599d022810fac45f0e P b2b871392e9945ef93d6708b940f10e6: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260812 06:55:32.025671 29031 tablet_bootstrap.cc:492] T 2cf091137a0f49599d022810fac45f0e P 2cebfda268094256bb864f37e363b58c: No bootstrap required, opened a new log
I20260812 06:55:32.025928 29031 ts_tablet_manager.cc:1397] T 2cf091137a0f49599d022810fac45f0e P 2cebfda268094256bb864f37e363b58c: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260812 06:55:32.025749 29032 raft_consensus.cc:348] T 2cf091137a0f49599d022810fac45f0e P a2cecf39b4004e88aadc9378c23aa474 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2cebfda268094256bb864f37e363b58c" member_type: VOTER last_known_addr { host: "127.19.52.65" port: 34495 } } peers { permanent_uuid: "b2b871392e9945ef93d6708b940f10e6" member_type: VOTER last_known_addr { host: "127.19.52.66" port: 38373 } } peers { permanent_uuid: "a2cecf39b4004e88aadc9378c23aa474" member_type: VOTER last_known_addr { host: "127.19.52.67" port: 44615 } }
I20260812 06:55:32.026173 29032 raft_consensus.cc:374] T 2cf091137a0f49599d022810fac45f0e P a2cecf39b4004e88aadc9378c23aa474 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 06:55:32.026099 29033 raft_consensus.cc:348] T 2cf091137a0f49599d022810fac45f0e P b2b871392e9945ef93d6708b940f10e6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2cebfda268094256bb864f37e363b58c" member_type: VOTER last_known_addr { host: "127.19.52.65" port: 34495 } } peers { permanent_uuid: "b2b871392e9945ef93d6708b940f10e6" member_type: VOTER last_known_addr { host: "127.19.52.66" port: 38373 } } peers { permanent_uuid: "a2cecf39b4004e88aadc9378c23aa474" member_type: VOTER last_known_addr { host: "127.19.52.67" port: 44615 } }
I20260812 06:55:32.026278 29032 raft_consensus.cc:729] T 2cf091137a0f49599d022810fac45f0e P a2cecf39b4004e88aadc9378c23aa474 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a2cecf39b4004e88aadc9378c23aa474, State: Initialized, Role: FOLLOWER
I20260812 06:55:32.026372 29033 raft_consensus.cc:374] T 2cf091137a0f49599d022810fac45f0e P b2b871392e9945ef93d6708b940f10e6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 06:55:32.026463 29033 raft_consensus.cc:729] T 2cf091137a0f49599d022810fac45f0e P b2b871392e9945ef93d6708b940f10e6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b2b871392e9945ef93d6708b940f10e6, State: Initialized, Role: FOLLOWER
I20260812 06:55:32.026681 29032 consensus_queue.cc:260] T 2cf091137a0f49599d022810fac45f0e P a2cecf39b4004e88aadc9378c23aa474 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2cebfda268094256bb864f37e363b58c" member_type: VOTER last_known_addr { host: "127.19.52.65" port: 34495 } } peers { permanent_uuid: "b2b871392e9945ef93d6708b940f10e6" member_type: VOTER last_known_addr { host: "127.19.52.66" port: 38373 } } peers { permanent_uuid: "a2cecf39b4004e88aadc9378c23aa474" member_type: VOTER last_known_addr { host: "127.19.52.67" port: 44615 } }
I20260812 06:55:32.027115 29031 raft_consensus.cc:348] T 2cf091137a0f49599d022810fac45f0e P 2cebfda268094256bb864f37e363b58c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2cebfda268094256bb864f37e363b58c" member_type: VOTER last_known_addr { host: "127.19.52.65" port: 34495 } } peers { permanent_uuid: "b2b871392e9945ef93d6708b940f10e6" member_type: VOTER last_known_addr { host: "127.19.52.66" port: 38373 } } peers { permanent_uuid: "a2cecf39b4004e88aadc9378c23aa474" member_type: VOTER last_known_addr { host: "127.19.52.67" port: 44615 } }
I20260812 06:55:32.027359 29031 raft_consensus.cc:374] T 2cf091137a0f49599d022810fac45f0e P 2cebfda268094256bb864f37e363b58c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 06:55:32.027458 29031 raft_consensus.cc:729] T 2cf091137a0f49599d022810fac45f0e P 2cebfda268094256bb864f37e363b58c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2cebfda268094256bb864f37e363b58c, State: Initialized, Role: FOLLOWER
I20260812 06:55:32.027513 29032 ts_tablet_manager.cc:1428] T 2cf091137a0f49599d022810fac45f0e P a2cecf39b4004e88aadc9378c23aa474: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260812 06:55:32.027390 29033 consensus_queue.cc:260] T 2cf091137a0f49599d022810fac45f0e P b2b871392e9945ef93d6708b940f10e6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2cebfda268094256bb864f37e363b58c" member_type: VOTER last_known_addr { host: "127.19.52.65" port: 34495 } } peers { permanent_uuid: "b2b871392e9945ef93d6708b940f10e6" member_type: VOTER last_known_addr { host: "127.19.52.66" port: 38373 } } peers { permanent_uuid: "a2cecf39b4004e88aadc9378c23aa474" member_type: VOTER last_known_addr { host: "127.19.52.67" port: 44615 } }
I20260812 06:55:32.027727 29031 consensus_queue.cc:260] T 2cf091137a0f49599d022810fac45f0e P 2cebfda268094256bb864f37e363b58c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2cebfda268094256bb864f37e363b58c" member_type: VOTER last_known_addr { host: "127.19.52.65" port: 34495 } } peers { permanent_uuid: "b2b871392e9945ef93d6708b940f10e6" member_type: VOTER last_known_addr { host: "127.19.52.66" port: 38373 } } peers { permanent_uuid: "a2cecf39b4004e88aadc9378c23aa474" member_type: VOTER last_known_addr { host: "127.19.52.67" port: 44615 } }
I20260812 06:55:32.029158 29033 ts_tablet_manager.cc:1428] T 2cf091137a0f49599d022810fac45f0e P b2b871392e9945ef93d6708b940f10e6: Time spent starting tablet: real 0.004s user 0.002s sys 0.000s
I20260812 06:55:32.029762 29031 ts_tablet_manager.cc:1428] T 2cf091137a0f49599d022810fac45f0e P 2cebfda268094256bb864f37e363b58c: Time spent starting tablet: real 0.004s user 0.003s sys 0.000s
I20260812 06:55:32.204737 29045 raft_consensus.cc:482] T 2cf091137a0f49599d022810fac45f0e P b2b871392e9945ef93d6708b940f10e6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 06:55:32.205016 29045 raft_consensus.cc:504] T 2cf091137a0f49599d022810fac45f0e P b2b871392e9945ef93d6708b940f10e6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2cebfda268094256bb864f37e363b58c" member_type: VOTER last_known_addr { host: "127.19.52.65" port: 34495 } } peers { permanent_uuid: "b2b871392e9945ef93d6708b940f10e6" member_type: VOTER last_known_addr { host: "127.19.52.66" port: 38373 } } peers { permanent_uuid: "a2cecf39b4004e88aadc9378c23aa474" member_type: VOTER last_known_addr { host: "127.19.52.67" port: 44615 } }
I20260812 06:55:32.206455 29045 leader_election.cc:288] T 2cf091137a0f49599d022810fac45f0e P b2b871392e9945ef93d6708b940f10e6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2cebfda268094256bb864f37e363b58c (127.19.52.65:34495), a2cecf39b4004e88aadc9378c23aa474 (127.19.52.67:44615)
I20260812 06:55:32.206877 28628 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2cf091137a0f49599d022810fac45f0e" candidate_uuid: "b2b871392e9945ef93d6708b940f10e6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2cebfda268094256bb864f37e363b58c" is_pre_election: true
I20260812 06:55:32.207201 28628 raft_consensus.cc:2454] T 2cf091137a0f49599d022810fac45f0e P 2cebfda268094256bb864f37e363b58c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b2b871392e9945ef93d6708b940f10e6 in term 0.
I20260812 06:55:32.207221 28945 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2cf091137a0f49599d022810fac45f0e" candidate_uuid: "b2b871392e9945ef93d6708b940f10e6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2cecf39b4004e88aadc9378c23aa474" is_pre_election: true
I20260812 06:55:32.207463 28945 raft_consensus.cc:2454] T 2cf091137a0f49599d022810fac45f0e P a2cecf39b4004e88aadc9378c23aa474 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b2b871392e9945ef93d6708b940f10e6 in term 0.
I20260812 06:55:32.207897 28707 leader_election.cc:302] T 2cf091137a0f49599d022810fac45f0e P b2b871392e9945ef93d6708b940f10e6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2cebfda268094256bb864f37e363b58c, b2b871392e9945ef93d6708b940f10e6; no voters:
I20260812 06:55:32.208236 29045 raft_consensus.cc:2781] T 2cf091137a0f49599d022810fac45f0e P b2b871392e9945ef93d6708b940f10e6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 06:55:32.208338 29045 raft_consensus.cc:482] T 2cf091137a0f49599d022810fac45f0e P b2b871392e9945ef93d6708b940f10e6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 06:55:32.208401 29045 raft_consensus.cc:3037] T 2cf091137a0f49599d022810fac45f0e P b2b871392e9945ef93d6708b940f10e6 [term 0 FOLLOWER]: Advancing to term 1
I20260812 06:55:32.210377 29045 raft_consensus.cc:504] T 2cf091137a0f49599d022810fac45f0e P b2b871392e9945ef93d6708b940f10e6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2cebfda268094256bb864f37e363b58c" member_type: VOTER last_known_addr { host: "127.19.52.65" port: 34495 } } peers { permanent_uuid: "b2b871392e9945ef93d6708b940f10e6" member_type: VOTER last_known_addr { host: "127.19.52.66" port: 38373 } } peers { permanent_uuid: "a2cecf39b4004e88aadc9378c23aa474" member_type: VOTER last_known_addr { host: "127.19.52.67" port: 44615 } }
I20260812 06:55:32.211079 29045 leader_election.cc:288] T 2cf091137a0f49599d022810fac45f0e P b2b871392e9945ef93d6708b940f10e6 [CANDIDATE]: Term 1 election: Requested vote from peers 2cebfda268094256bb864f37e363b58c (127.19.52.65:34495), a2cecf39b4004e88aadc9378c23aa474 (127.19.52.67:44615)
I20260812 06:55:32.211807 28628 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2cf091137a0f49599d022810fac45f0e" candidate_uuid: "b2b871392e9945ef93d6708b940f10e6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2cebfda268094256bb864f37e363b58c"
I20260812 06:55:32.211900 28945 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2cf091137a0f49599d022810fac45f0e" candidate_uuid: "b2b871392e9945ef93d6708b940f10e6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2cecf39b4004e88aadc9378c23aa474"
I20260812 06:55:32.212075 28628 raft_consensus.cc:3037] T 2cf091137a0f49599d022810fac45f0e P 2cebfda268094256bb864f37e363b58c [term 0 FOLLOWER]: Advancing to term 1
I20260812 06:55:32.212157 28945 raft_consensus.cc:3037] T 2cf091137a0f49599d022810fac45f0e P a2cecf39b4004e88aadc9378c23aa474 [term 0 FOLLOWER]: Advancing to term 1
I20260812 06:55:32.213961 28945 raft_consensus.cc:2454] T 2cf091137a0f49599d022810fac45f0e P a2cecf39b4004e88aadc9378c23aa474 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b2b871392e9945ef93d6708b940f10e6 in term 1.
I20260812 06:55:32.213981 28628 raft_consensus.cc:2454] T 2cf091137a0f49599d022810fac45f0e P 2cebfda268094256bb864f37e363b58c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b2b871392e9945ef93d6708b940f10e6 in term 1.
I20260812 06:55:32.214569 28709 leader_election.cc:302] T 2cf091137a0f49599d022810fac45f0e P b2b871392e9945ef93d6708b940f10e6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2cecf39b4004e88aadc9378c23aa474, b2b871392e9945ef93d6708b940f10e6; no voters:
I20260812 06:55:32.214903 29045 raft_consensus.cc:2781] T 2cf091137a0f49599d022810fac45f0e P b2b871392e9945ef93d6708b940f10e6 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 06:55:32.215263 29045 raft_consensus.cc:686] T 2cf091137a0f49599d022810fac45f0e P b2b871392e9945ef93d6708b940f10e6 [term 1 LEADER]: Becoming Leader. State: Replica: b2b871392e9945ef93d6708b940f10e6, State: Running, Role: LEADER
I20260812 06:55:32.215715 29045 consensus_queue.cc:237] T 2cf091137a0f49599d022810fac45f0e P b2b871392e9945ef93d6708b940f10e6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2cebfda268094256bb864f37e363b58c" member_type: VOTER last_known_addr { host: "127.19.52.65" port: 34495 } } peers { permanent_uuid: "b2b871392e9945ef93d6708b940f10e6" member_type: VOTER last_known_addr { host: "127.19.52.66" port: 38373 } } peers { permanent_uuid: "a2cecf39b4004e88aadc9378c23aa474" member_type: VOTER last_known_addr { host: "127.19.52.67" port: 44615 } }
I20260812 06:55:32.219834 28449 catalog_manager.cc:5495] T 2cf091137a0f49599d022810fac45f0e P b2b871392e9945ef93d6708b940f10e6 reported cstate change: term changed from 0 to 1, leader changed from <none> to b2b871392e9945ef93d6708b940f10e6 (127.19.52.66). New cstate: current_term: 1 leader_uuid: "b2b871392e9945ef93d6708b940f10e6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2cebfda268094256bb864f37e363b58c" member_type: VOTER last_known_addr { host: "127.19.52.65" port: 34495 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b2b871392e9945ef93d6708b940f10e6" member_type: VOTER last_known_addr { host: "127.19.52.66" port: 38373 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a2cecf39b4004e88aadc9378c23aa474" member_type: VOTER last_known_addr { host: "127.19.52.67" port: 44615 } health_report { overall_health: UNKNOWN } } }
I20260812 06:55:32.263055 29126 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:32.255219 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58904 (local address 127.19.52.66:38373)
0812 06:55:32.255746 (+ 527us) server_negotiation.cc:206] Beginning negotiation
0812 06:55:32.255753 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 06:55:32.255898 (+ 145us) server_negotiation.cc:407] Connection header received
0812 06:55:32.256263 (+ 365us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:55:32.256267 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:55:32.256616 (+ 349us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:55:32.256801 (+ 185us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 06:55:32.257235 (+ 434us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:32.258635 (+ 1400us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:32.260314 (+ 1679us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:32.261052 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:32.261269 (+ 217us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:55:32.261730 (+ 461us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 06:55:32.262047 (+ 317us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 06:55:32.262190 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0812 06:55:32.262584 (+ 394us) server_negotiation.cc:299] Negotiation successful
0812 06:55:32.262701 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":348,"thread_start_us":208,"threads_started":1}
I20260812 06:55:32.281214 29130 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:32.275362 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.66:38373 (local address 127.19.52.66:52279)
0812 06:55:32.275885 (+ 523us) negotiation.cc:107] Waiting for socket to connect
0812 06:55:32.275908 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0812 06:55:32.276081 (+ 173us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:55:32.276422 (+ 341us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:55:32.276424 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:55:32.276438 (+ 14us) client_negotiation.cc:189] Negotiated authn=TOKEN
0812 06:55:32.276643 (+ 205us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:55:32.276650 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:55:32.277960 (+ 1310us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:55:32.277964 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:55:32.278979 (+ 1015us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:55:32.278998 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:55:32.279764 (+ 766us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:55:32.279767 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:55:32.279862 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:55:32.279963 (+ 101us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0812 06:55:32.280671 (+ 708us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0812 06:55:32.280678 (+ 7us) client_negotiation.cc:769] Sending connection context
0812 06:55:32.280804 (+ 126us) client_negotiation.cc:240] Negotiation successful
0812 06:55:32.280919 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":297,"thread_start_us":161,"threads_started":1}
I20260812 06:55:32.281210 29126 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:32.275509 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.52.66:52279 (local address 127.19.52.66:38373)
0812 06:55:32.275696 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0812 06:55:32.275699 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 06:55:32.276045 (+ 346us) server_negotiation.cc:407] Connection header received
0812 06:55:32.276228 (+ 183us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:55:32.276231 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:55:32.276281 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:55:32.276393 (+ 112us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 06:55:32.276797 (+ 404us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:32.277700 (+ 903us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:32.279197 (+ 1497us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:32.279576 (+ 379us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:32.279733 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:55:32.280151 (+ 418us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 06:55:32.280315 (+ 164us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 06:55:32.280518 (+ 203us) server_negotiation.cc:1035] Waiting for connection context
0812 06:55:32.280866 (+ 348us) server_negotiation.cc:299] Negotiation successful
0812 06:55:32.280978 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":64}
I20260812 06:55:32.292397 28628 raft_consensus.cc:1261] T 2cf091137a0f49599d022810fac45f0e P 2cebfda268094256bb864f37e363b58c [term 1 FOLLOWER]: Refusing update from remote peer b2b871392e9945ef93d6708b940f10e6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 06:55:32.292397 28945 raft_consensus.cc:1261] T 2cf091137a0f49599d022810fac45f0e P a2cecf39b4004e88aadc9378c23aa474 [term 1 FOLLOWER]: Refusing update from remote peer b2b871392e9945ef93d6708b940f10e6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 06:55:32.293313 29045 consensus_queue.cc:1035] T 2cf091137a0f49599d022810fac45f0e P b2b871392e9945ef93d6708b940f10e6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2cecf39b4004e88aadc9378c23aa474" member_type: VOTER last_known_addr { host: "127.19.52.67" port: 44615 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 06:55:32.293598 29042 consensus_queue.cc:1035] T 2cf091137a0f49599d022810fac45f0e P b2b871392e9945ef93d6708b940f10e6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2cebfda268094256bb864f37e363b58c" member_type: VOTER last_known_addr { host: "127.19.52.65" port: 34495 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 06:55:32.358134 28449 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42116:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 06:55:32.358460 28449 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42116:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 06:55:32.362397 28449 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 3bce2c30f41c4196af2accc6509b44ff: Sending DeleteTablet for 3 replicas of tablet 2cf091137a0f49599d022810fac45f0e
I20260812 06:55:32.363940 28763 tablet_service.cc:1514] Processing DeleteTablet for tablet 2cf091137a0f49599d022810fac45f0e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 06:55:32 UTC) from {username='oryx', principal='oryx/127.19.52.126@KRBTEST.COM'} at 127.0.0.1:58876
I20260812 06:55:32.364084 28607 tablet_service.cc:1514] Processing DeleteTablet for tablet 2cf091137a0f49599d022810fac45f0e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 06:55:32 UTC) from {username='oryx', principal='oryx/127.19.52.126@KRBTEST.COM'} at 127.0.0.1:53332
I20260812 06:55:32.364346 29062 tablet_replica.cc:330] T 2cf091137a0f49599d022810fac45f0e P b2b871392e9945ef93d6708b940f10e6: stopping tablet replica
I20260812 06:55:32.364347 28922 tablet_service.cc:1514] Processing DeleteTablet for tablet 2cf091137a0f49599d022810fac45f0e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 06:55:32 UTC) from {username='oryx', principal='oryx/127.19.52.126@KRBTEST.COM'} at 127.0.0.1:59064
I20260812 06:55:32.364524 29060 tablet_replica.cc:330] T 2cf091137a0f49599d022810fac45f0e P 2cebfda268094256bb864f37e363b58c: stopping tablet replica
I20260812 06:55:32.364634 29062 raft_consensus.cc:2229] T 2cf091137a0f49599d022810fac45f0e P b2b871392e9945ef93d6708b940f10e6 [term 1 LEADER]: Raft consensus shutting down.
I20260812 06:55:32.364727 29100 tablet_replica.cc:330] T 2cf091137a0f49599d022810fac45f0e P a2cecf39b4004e88aadc9378c23aa474: stopping tablet replica
I20260812 06:55:32.364755 29060 raft_consensus.cc:2229] T 2cf091137a0f49599d022810fac45f0e P 2cebfda268094256bb864f37e363b58c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 06:55:32.364955 29100 raft_consensus.cc:2229] T 2cf091137a0f49599d022810fac45f0e P a2cecf39b4004e88aadc9378c23aa474 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 06:55:32.365005 29060 raft_consensus.cc:2258] T 2cf091137a0f49599d022810fac45f0e P 2cebfda268094256bb864f37e363b58c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 06:55:32.365007 29062 raft_consensus.cc:2258] T 2cf091137a0f49599d022810fac45f0e P b2b871392e9945ef93d6708b940f10e6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 06:55:32.365192 29100 raft_consensus.cc:2258] T 2cf091137a0f49599d022810fac45f0e P a2cecf39b4004e88aadc9378c23aa474 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 06:55:32.366972 29100 ts_tablet_manager.cc:1905] T 2cf091137a0f49599d022810fac45f0e P a2cecf39b4004e88aadc9378c23aa474: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 06:55:32.367002 29060 ts_tablet_manager.cc:1905] T 2cf091137a0f49599d022810fac45f0e P 2cebfda268094256bb864f37e363b58c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 06:55:32.367030 29062 ts_tablet_manager.cc:1905] T 2cf091137a0f49599d022810fac45f0e P b2b871392e9945ef93d6708b940f10e6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 06:55:32.368363 19665 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu with pid 28531
I20260812 06:55:32.379189 29100 ts_tablet_manager.cc:1918] T 2cf091137a0f49599d022810fac45f0e P a2cecf39b4004e88aadc9378c23aa474: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260812 06:55:32.379189 29062 ts_tablet_manager.cc:1918] T 2cf091137a0f49599d022810fac45f0e P b2b871392e9945ef93d6708b940f10e6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260812 06:55:32.379384 29062 log.cc:1196] T 2cf091137a0f49599d022810fac45f0e P b2b871392e9945ef93d6708b940f10e6: Deleting WAL directory at /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786517687384214-19665-0/minicluster-data/ts-1/wal/wals/2cf091137a0f49599d022810fac45f0e
I20260812 06:55:32.379760 29100 log.cc:1196] T 2cf091137a0f49599d022810fac45f0e P a2cecf39b4004e88aadc9378c23aa474: Deleting WAL directory at /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786517687384214-19665-0/minicluster-data/ts-2/wal/wals/2cf091137a0f49599d022810fac45f0e
I20260812 06:55:32.380100 29062 ts_tablet_manager.cc:1939] T 2cf091137a0f49599d022810fac45f0e P b2b871392e9945ef93d6708b940f10e6: Deleting consensus metadata
I20260812 06:55:32.380405 29100 ts_tablet_manager.cc:1939] T 2cf091137a0f49599d022810fac45f0e P a2cecf39b4004e88aadc9378c23aa474: Deleting consensus metadata
I20260812 06:55:32.381896 28434 catalog_manager.cc:4841] TS a2cecf39b4004e88aadc9378c23aa474 (127.19.52.67:44615): tablet 2cf091137a0f49599d022810fac45f0e (table test-table [id=9f0d9a94819a4f1aa6c1f633289ed7ae]) successfully deleted
I20260812 06:55:32.381950 28433 catalog_manager.cc:4841] TS b2b871392e9945ef93d6708b940f10e6 (127.19.52.66:38373): tablet 2cf091137a0f49599d022810fac45f0e (table test-table [id=9f0d9a94819a4f1aa6c1f633289ed7ae]) successfully deleted
W20260812 06:55:32.384783 28433 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.19.52.65:34495 (error 108)
I20260812 06:55:32.385084 19665 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu with pid 28686
I20260812 06:55:32.391031 29136 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:32.389846 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.65:34495 (local address 127.0.0.1:53342)
0812 06:55:32.390482 (+ 636us) negotiation.cc:107] Waiting for socket to connect
0812 06:55:32.390778 (+ 296us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.52.65:34495: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":360,"thread_start_us":129,"threads_started":1}
W20260812 06:55:32.392127 28433 catalog_manager.cc:4584] TS 2cebfda268094256bb864f37e363b58c (127.19.52.65:34495): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 2cf091137a0f49599d022810fac45f0e: Network error: Client connection negotiation failed: client connection to 127.19.52.65:34495: connect: Connection refused (error 111)
I20260812 06:55:32.404450 19665 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu with pid 28851
I20260812 06:55:32.422111 19665 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu with pid 28412
2026-08-12T06:55:32Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipal (3721 ms)
[ RUN ] SecurityITest.TestNonExistentPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786517687384214-19665-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 06:55:32 dist-test-slave-65mx krb5kdc[29139](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 06:55:32 dist-test-slave-65mx krb5kdc[29139](info): set up 2 sockets
Aug 12 06:55:32 dist-test-slave-65mx krb5kdc[29139](info): commencing operation
krb5kdc: starting...
W20260812 06:55:33.724715 19665 mini_kdc.cc:121] Time spent starting KDC: real 1.264s user 0.000s sys 0.013s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786517687384214-19665-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786517687384214-19665-0/krb5kdc/test-user.keytab.
Aug 12 06:55:33 dist-test-slave-65mx krb5kdc[29139](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517733, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T06:55:33Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T06:55:33Z Disabled control of system clock
WARNING: no policy specified for kudu/127.19.52.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.52.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.52.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786517687384214-19665-0/krb5kdc/kudu_127.19.52.126.keytab.
Entry for principal kudu/127.19.52.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786517687384214-19665-0/krb5kdc/kudu_127.19.52.126.keytab.
I20260812 06:55:33.869150 19665 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu
/tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786517687384214-19665-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786517687384214-19665-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786517687384214-19665-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786517687384214-19665-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.52.126:44687
--webserver_interface=127.19.52.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786517687384214-19665-0/krb5kdc/kudu_127.19.52.126.keytab
--principal=kudu/127.19.52.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.52.84:46529
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.19.52.126:44687
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--principal=oryx with env {KRB5CCNAME=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786517687384214-19665-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786517687384214-19665-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786517687384214-19665-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 06:55:33.992647 29152 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:55:33.993005 29152 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:55:33.993088 29152 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 06:55:33.997399 29152 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 06:55:33.997498 29152 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 06:55:33.997538 29152 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 06:55:33.997576 29152 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 06:55:33.997611 29152 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 06:55:34.003182 29152 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.52.84:46529
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786517687384214-19665-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786517687384214-19665-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.19.52.126:44687
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786517687384214-19665-0/krb5kdc/kudu_127.19.52.126.keytab
--principal=oryx
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.52.126:44687
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786517687384214-19665-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.19.52.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29152
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786517687384214-19665-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 06:15:54 UTC on ccycloud.quasar-rnfzce.root.comops.site
build id 4581989
I20260812 06:55:34.004582 29152 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:55:34.005995 29152 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 06:55:34.014891 29157 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:55:34.014889 29160 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:55:34.015163 29152 server_base.cc:1034] running on GCE node
W20260812 06:55:34.014890 29158 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:55:34.015803 29152 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:55:34.017133 29152 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:55:34.018362 29152 hybrid_clock.cc:648] HybridClock initialized: now 1786517734018319 us; error 55 us; skew 500 ppm
Runtime error: kinit failed: unable to login from keytab: Keytab contains no suitable keys for oryx@KRBTEST.COM
2026-08-12T06:55:34Z chronyd exiting
[ OK ] SecurityITest.TestNonExistentPrincipal (1592 ms)
[ RUN ] SecurityITest.TestMismatchingPrincipals
Loading random data
Initializing database '/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517687384214-19665-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 06:55:34 dist-test-slave-65mx krb5kdc[29165](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 06:55:34 dist-test-slave-65mx krb5kdc[29165](info): set up 2 sockets
Aug 12 06:55:34 dist-test-slave-65mx krb5kdc[29165](info): commencing operation
krb5kdc: starting...
W20260812 06:55:35.167969 19665 mini_kdc.cc:121] Time spent starting KDC: real 1.116s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517687384214-19665-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517687384214-19665-0/krb5kdc/test-user.keytab.
Aug 12 06:55:35 dist-test-slave-65mx krb5kdc[29165](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517735, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T06:55:35Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T06:55:35Z Disabled control of system clock
WARNING: no policy specified for kudu/127.19.52.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.52.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.52.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517687384214-19665-0/krb5kdc/kudu_127.19.52.126.keytab.
Entry for principal kudu/127.19.52.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517687384214-19665-0/krb5kdc/kudu_127.19.52.126.keytab.
I20260812 06:55:35.316838 19665 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu
/tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517687384214-19665-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517687384214-19665-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517687384214-19665-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517687384214-19665-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.52.126:44739
--webserver_interface=127.19.52.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517687384214-19665-0/krb5kdc/kudu_127.19.52.126.keytab
--principal=kudu/127.19.52.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.52.84:43735
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.19.52.126:44739
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517687384214-19665-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517687384214-19665-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517687384214-19665-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 06:55:35.457048 29180 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:55:35.457376 29180 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:55:35.457420 29180 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 06:55:35.461812 29180 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 06:55:35.461906 29180 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 06:55:35.461931 29180 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 06:55:35.461949 29180 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 06:55:35.461966 29180 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 06:55:35.467857 29180 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.52.84:43735
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517687384214-19665-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517687384214-19665-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.19.52.126:44739
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517687384214-19665-0/krb5kdc/kudu_127.19.52.126.keytab
--principal=kudu/127.19.52.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.52.126:44739
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517687384214-19665-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.19.52.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29180
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517687384214-19665-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 06:15:54 UTC on ccycloud.quasar-rnfzce.root.comops.site
build id 4581989
I20260812 06:55:35.469203 29180 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:55:35.470305 29180 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 06:55:35.480036 29191 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:55:35.480047 29187 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:55:35.480034 29188 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:55:35.480729 29180 server_base.cc:1034] running on GCE node
I20260812 06:55:35.481483 29180 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:55:35.482795 29180 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:55:35.484048 29180 hybrid_clock.cc:648] HybridClock initialized: now 1786517735484014 us; error 50 us; skew 500 ppm
Aug 12 06:55:35 dist-test-slave-65mx krb5kdc[29165](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517735, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.52.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 06:55:35.487365 29180 init.cc:377] Logged in from keytab as kudu/127.19.52.126@KRBTEST.COM (short username kudu)
I20260812 06:55:35.488494 29180 webserver.cc:460] Webserver started at http://127.19.52.126:34635/ using document root <none> and password file <none>
I20260812 06:55:35.489116 29180 fs_manager.cc:359] Metadata directory not provided
I20260812 06:55:35.489216 29180 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:55:35.489459 29180 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 06:55:35.492026 29180 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517687384214-19665-0/minicluster-data/master-0/data/instance:
uuid: "93edba2c8aca4bee8e1e0869d33fdb59"
format_stamp: "Formatted at 2026-08-12 06:55:35 on dist-test-slave-65mx"
server_key: "c142b935a9cb1ef8912af34cb9397ade"
server_key_iv: "cc33419d8afc8243f7e0b8cc7f0c2da3"
server_key_version: "encryptionkey@0"
I20260812 06:55:35.492748 29180 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517687384214-19665-0/minicluster-data/master-0/wal/instance:
uuid: "93edba2c8aca4bee8e1e0869d33fdb59"
format_stamp: "Formatted at 2026-08-12 06:55:35 on dist-test-slave-65mx"
server_key: "c142b935a9cb1ef8912af34cb9397ade"
server_key_iv: "cc33419d8afc8243f7e0b8cc7f0c2da3"
server_key_version: "encryptionkey@0"
I20260812 06:55:35.497483 29180 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260812 06:55:35.500705 29199 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 06:55:35.502486 29180 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 06:55:35.502702 29180 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517687384214-19665-0/minicluster-data/master-0/data,/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517687384214-19665-0/minicluster-data/master-0/wal
uuid: "93edba2c8aca4bee8e1e0869d33fdb59"
format_stamp: "Formatted at 2026-08-12 06:55:35 on dist-test-slave-65mx"
server_key: "c142b935a9cb1ef8912af34cb9397ade"
server_key_iv: "cc33419d8afc8243f7e0b8cc7f0c2da3"
server_key_version: "encryptionkey@0"
I20260812 06:55:35.502861 29180 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517687384214-19665-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517687384214-19665-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517687384214-19665-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:55:35.531983 29180 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 06:55:35.535974 29180 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 06:55:35.536283 29180 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:55:35.544914 29180 rpc_server.cc:307] RPC server started. Bound to: 127.19.52.126:44739
I20260812 06:55:35.544945 29255 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.52.126:44739 every 8 connection(s)
I20260812 06:55:35.546039 29180 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517687384214-19665-0/minicluster-data/master-0/data/info.pb
I20260812 06:55:35.549206 29256 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 06:55:35.555953 29256 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 93edba2c8aca4bee8e1e0869d33fdb59: Bootstrap starting.
I20260812 06:55:35.556047 19665 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu as pid 29180
I20260812 06:55:35.556180 19665 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517687384214-19665-0/minicluster-data/master-0/wal/instance
I20260812 06:55:35.556520 19665 external_mini_cluster.cc:1428] Setting key eb68931f83e134d2bb00d966931350f4
I20260812 06:55:35.559212 29256 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 93edba2c8aca4bee8e1e0869d33fdb59: Neither blocks nor log segments found. Creating new log.
I20260812 06:55:35.560077 29256 log.cc:824] T 00000000000000000000000000000000 P 93edba2c8aca4bee8e1e0869d33fdb59: Log is configured to *not* fsync() on all Append() calls
I20260812 06:55:35.562493 29256 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 93edba2c8aca4bee8e1e0869d33fdb59: No bootstrap required, opened a new log
I20260812 06:55:35.567061 29256 raft_consensus.cc:348] T 00000000000000000000000000000000 P 93edba2c8aca4bee8e1e0869d33fdb59 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "93edba2c8aca4bee8e1e0869d33fdb59" member_type: VOTER last_known_addr { host: "127.19.52.126" port: 44739 } }
I20260812 06:55:35.567462 29256 raft_consensus.cc:374] T 00000000000000000000000000000000 P 93edba2c8aca4bee8e1e0869d33fdb59 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 06:55:35.567523 29256 raft_consensus.cc:729] T 00000000000000000000000000000000 P 93edba2c8aca4bee8e1e0869d33fdb59 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 93edba2c8aca4bee8e1e0869d33fdb59, State: Initialized, Role: FOLLOWER
Aug 12 06:55:35 dist-test-slave-65mx krb5kdc[29165](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517735, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.19.52.126@KRBTEST.COM
I20260812 06:55:35.568122 29256 consensus_queue.cc:260] T 00000000000000000000000000000000 P 93edba2c8aca4bee8e1e0869d33fdb59 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "93edba2c8aca4bee8e1e0869d33fdb59" member_type: VOTER last_known_addr { host: "127.19.52.126" port: 44739 } }
I20260812 06:55:35.568333 29256 raft_consensus.cc:388] T 00000000000000000000000000000000 P 93edba2c8aca4bee8e1e0869d33fdb59 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 06:55:35.568403 29256 raft_consensus.cc:482] T 00000000000000000000000000000000 P 93edba2c8aca4bee8e1e0869d33fdb59 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 06:55:35.568522 29256 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 93edba2c8aca4bee8e1e0869d33fdb59 [term 0 FOLLOWER]: Advancing to term 1
I20260812 06:55:35.570609 29256 raft_consensus.cc:504] T 00000000000000000000000000000000 P 93edba2c8aca4bee8e1e0869d33fdb59 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "93edba2c8aca4bee8e1e0869d33fdb59" member_type: VOTER last_known_addr { host: "127.19.52.126" port: 44739 } }
I20260812 06:55:35.571249 29256 leader_election.cc:302] T 00000000000000000000000000000000 P 93edba2c8aca4bee8e1e0869d33fdb59 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 93edba2c8aca4bee8e1e0869d33fdb59; no voters:
I20260812 06:55:35.572180 29256 leader_election.cc:288] T 00000000000000000000000000000000 P 93edba2c8aca4bee8e1e0869d33fdb59 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 06:55:35.572619 29262 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 93edba2c8aca4bee8e1e0869d33fdb59 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 06:55:35.572957 29262 raft_consensus.cc:686] T 00000000000000000000000000000000 P 93edba2c8aca4bee8e1e0869d33fdb59 [term 1 LEADER]: Becoming Leader. State: Replica: 93edba2c8aca4bee8e1e0869d33fdb59, State: Running, Role: LEADER
I20260812 06:55:35.573422 29262 consensus_queue.cc:237] T 00000000000000000000000000000000 P 93edba2c8aca4bee8e1e0869d33fdb59 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "93edba2c8aca4bee8e1e0869d33fdb59" member_type: VOTER last_known_addr { host: "127.19.52.126" port: 44739 } }
I20260812 06:55:35.573897 29256 sys_catalog.cc:564] T 00000000000000000000000000000000 P 93edba2c8aca4bee8e1e0869d33fdb59 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 06:55:35.575866 29264 sys_catalog.cc:455] T 00000000000000000000000000000000 P 93edba2c8aca4bee8e1e0869d33fdb59 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 93edba2c8aca4bee8e1e0869d33fdb59. Latest consensus state: current_term: 1 leader_uuid: "93edba2c8aca4bee8e1e0869d33fdb59" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "93edba2c8aca4bee8e1e0869d33fdb59" member_type: VOTER last_known_addr { host: "127.19.52.126" port: 44739 } } }
I20260812 06:55:35.576063 29264 sys_catalog.cc:458] T 00000000000000000000000000000000 P 93edba2c8aca4bee8e1e0869d33fdb59 [sys.catalog]: This master's current role is: LEADER
I20260812 06:55:35.576973 29259 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:35.558542 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39394 (local address 127.19.52.126:44739)
0812 06:55:35.559093 (+ 551us) server_negotiation.cc:206] Beginning negotiation
0812 06:55:35.559111 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0812 06:55:35.559151 (+ 40us) server_negotiation.cc:407] Connection header received
0812 06:55:35.559955 (+ 804us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:55:35.559977 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:55:35.560403 (+ 426us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:55:35.560848 (+ 445us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:55:35.561776 (+ 928us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:35.563203 (+ 1427us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:35.564520 (+ 1317us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:35.565293 (+ 773us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:35.565514 (+ 221us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:55:35.568779 (+ 3265us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:55:35.568785 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:55:35.568800 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:55:35.568837 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:55:35.571282 (+ 2445us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:55:35.572457 (+ 1175us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:55:35.572467 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:55:35.572478 (+ 11us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:55:35.572707 (+ 229us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:55:35.573144 (+ 437us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:55:35.573148 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:55:35.573150 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:55:35.573902 (+ 752us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:55:35.574938 (+ 1036us) server_negotiation.cc:1035] Waiting for connection context
0812 06:55:35.575125 (+ 187us) server_negotiation.cc:299] Negotiation successful
0812 06:55:35.575696 (+ 571us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":362,"thread_start_us":163,"threads_started":1}
I20260812 06:55:35.576972 29263 sys_catalog.cc:455] T 00000000000000000000000000000000 P 93edba2c8aca4bee8e1e0869d33fdb59 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "93edba2c8aca4bee8e1e0869d33fdb59" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "93edba2c8aca4bee8e1e0869d33fdb59" member_type: VOTER last_known_addr { host: "127.19.52.126" port: 44739 } } }
I20260812 06:55:35.577289 29276 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 06:55:35.577569 29263 sys_catalog.cc:458] T 00000000000000000000000000000000 P 93edba2c8aca4bee8e1e0869d33fdb59 [sys.catalog]: This master's current role is: LEADER
I20260812 06:55:35.580706 29276 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 06:55:35.586710 29276 catalog_manager.cc:1347] Generated new cluster ID: 683b7c7b28f6489797e43d30d96f8a0d
I20260812 06:55:35.586887 29276 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 06:55:35.612399 29276 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 06:55:35.613543 29276 catalog_manager.cc:1504] Loading token signing keys...
I20260812 06:55:35.618853 29276 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 93edba2c8aca4bee8e1e0869d33fdb59: Generated new TSK 0
I20260812 06:55:35.619635 29276 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.19.52.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.52.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.52.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517687384214-19665-0/krb5kdc/kudu_127.19.52.65.keytab.
Entry for principal kudu/127.19.52.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517687384214-19665-0/krb5kdc/kudu_127.19.52.65.keytab.
I20260812 06:55:35.668344 19665 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu
/tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517687384214-19665-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517687384214-19665-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517687384214-19665-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517687384214-19665-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.52.65:0
--local_ip_for_outbound_sockets=127.19.52.65
--webserver_interface=127.19.52.65
--webserver_port=0
--tserver_master_addrs=127.19.52.126:44739
--keytab_file=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517687384214-19665-0/krb5kdc/kudu_127.19.52.65.keytab
--principal=kudu/127.19.52.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.52.84:43735
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517687384214-19665-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517687384214-19665-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517687384214-19665-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 06:55:35.798496 29283 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:55:35.799155 29283 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:55:35.799273 29283 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 06:55:35.804528 29283 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 06:55:35.804683 29283 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 06:55:35.804785 29283 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.52.65
I20260812 06:55:35.811441 29283 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.52.84:43735
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517687384214-19665-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517687384214-19665-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517687384214-19665-0/krb5kdc/kudu_127.19.52.65.keytab
--principal=kudu/127.19.52.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.52.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517687384214-19665-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.19.52.65
--webserver_port=0
--tserver_master_addrs=127.19.52.126:44739
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29283
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.52.65
--log_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517687384214-19665-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 06:15:54 UTC on ccycloud.quasar-rnfzce.root.comops.site
build id 4581989
I20260812 06:55:35.812763 29283 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:55:35.813900 29283 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 06:55:35.824486 29289 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:55:35.824621 29288 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:55:35.824678 29283 server_base.cc:1034] running on GCE node
W20260812 06:55:35.824554 29291 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:55:35.825256 29283 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:55:35.825990 29283 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:55:35.827194 29283 hybrid_clock.cc:648] HybridClock initialized: now 1786517735827172 us; error 36 us; skew 500 ppm
Aug 12 06:55:35 dist-test-slave-65mx krb5kdc[29165](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517735, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.52.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 06:55:35.830302 29283 init.cc:377] Logged in from keytab as kudu/127.19.52.65@KRBTEST.COM (short username kudu)
I20260812 06:55:35.831314 29283 webserver.cc:460] Webserver started at http://127.19.52.65:44731/ using document root <none> and password file <none>
I20260812 06:55:35.831907 29283 fs_manager.cc:359] Metadata directory not provided
I20260812 06:55:35.832010 29283 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:55:35.832262 29283 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 06:55:35.834493 29283 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517687384214-19665-0/minicluster-data/ts-0/data/instance:
uuid: "bc423acff98d45b79d8132e2a326d4a3"
format_stamp: "Formatted at 2026-08-12 06:55:35 on dist-test-slave-65mx"
server_key: "bef05a5cc98d6121a2f0a05d668ffece"
server_key_iv: "43745e12c92ea977af7360a2ecbd8517"
server_key_version: "encryptionkey@0"
I20260812 06:55:35.835129 29283 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517687384214-19665-0/minicluster-data/ts-0/wal/instance:
uuid: "bc423acff98d45b79d8132e2a326d4a3"
format_stamp: "Formatted at 2026-08-12 06:55:35 on dist-test-slave-65mx"
server_key: "bef05a5cc98d6121a2f0a05d668ffece"
server_key_iv: "43745e12c92ea977af7360a2ecbd8517"
server_key_version: "encryptionkey@0"
I20260812 06:55:35.839259 29283 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.004s
I20260812 06:55:35.842294 29300 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 06:55:35.843720 29283 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260812 06:55:35.843832 29283 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517687384214-19665-0/minicluster-data/ts-0/data,/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517687384214-19665-0/minicluster-data/ts-0/wal
uuid: "bc423acff98d45b79d8132e2a326d4a3"
format_stamp: "Formatted at 2026-08-12 06:55:35 on dist-test-slave-65mx"
server_key: "bef05a5cc98d6121a2f0a05d668ffece"
server_key_iv: "43745e12c92ea977af7360a2ecbd8517"
server_key_version: "encryptionkey@0"
I20260812 06:55:35.843933 29283 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517687384214-19665-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517687384214-19665-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517687384214-19665-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:55:35.872236 29283 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 06:55:35.876945 29283 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 06:55:35.877143 29283 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:55:35.877797 29283 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 06:55:35.878728 29283 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 06:55:35.878789 29283 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 06:55:35.878836 29283 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 06:55:35.878854 29283 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 06:55:35.889490 29283 rpc_server.cc:307] RPC server started. Bound to: 127.19.52.65:34467
I20260812 06:55:35.889530 29427 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.52.65:34467 every 8 connection(s)
I20260812 06:55:35.890545 29283 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517687384214-19665-0/minicluster-data/ts-0/data/info.pb
I20260812 06:55:35.898763 19665 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu as pid 29283
I20260812 06:55:35.898888 19665 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517687384214-19665-0/minicluster-data/ts-0/wal/instance
I20260812 06:55:35.899240 19665 external_mini_cluster.cc:1428] Setting key 94da7076e3a74b0b88da8a774ca5d4e4
Aug 12 06:55:35 dist-test-slave-65mx krb5kdc[29165](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517735, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.52.65@KRBTEST.COM for kudu/127.19.52.126@KRBTEST.COM
I20260812 06:55:35.905583 29259 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:35.892630 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.52.65:35497 (local address 127.19.52.126:44739)
0812 06:55:35.892777 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0812 06:55:35.892781 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 06:55:35.893981 (+ 1200us) server_negotiation.cc:407] Connection header received
0812 06:55:35.894879 (+ 898us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:55:35.894882 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:55:35.894952 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:55:35.895064 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:55:35.896291 (+ 1227us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:35.896890 (+ 599us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:35.898074 (+ 1184us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:35.898314 (+ 240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:35.898425 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:55:35.901494 (+ 3069us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:55:35.901497 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:55:35.901499 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:55:35.901523 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:55:35.903067 (+ 1544us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:55:35.903600 (+ 533us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:55:35.903605 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:55:35.903608 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:55:35.903708 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:55:35.904056 (+ 348us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:55:35.904061 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:55:35.904063 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:55:35.904322 (+ 259us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:55:35.904522 (+ 200us) server_negotiation.cc:1035] Waiting for connection context
0812 06:55:35.905148 (+ 626us) server_negotiation.cc:299] Negotiation successful
0812 06:55:35.905336 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260812 06:55:35.906423 29431 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:35.892981 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.126:44739 (local address 127.19.52.65:35497)
0812 06:55:35.893754 (+ 773us) negotiation.cc:107] Waiting for socket to connect
0812 06:55:35.893799 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0812 06:55:35.894666 (+ 867us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:55:35.895252 (+ 586us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:55:35.895259 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:55:35.895648 (+ 389us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:55:35.896134 (+ 486us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:55:35.896148 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:55:35.897039 (+ 891us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:55:35.897047 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:55:35.897936 (+ 889us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:55:35.897949 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:55:35.898518 (+ 569us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:55:35.898524 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:55:35.898681 (+ 157us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:55:35.899392 (+ 711us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 06:55:35.899408 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:55:35.901302 (+ 1894us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 06:55:35.903196 (+ 1894us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:55:35.903201 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:55:35.903214 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:55:35.903494 (+ 280us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:55:35.903805 (+ 311us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:55:35.903807 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:55:35.903808 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:55:35.903946 (+ 138us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:55:35.904468 (+ 522us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 06:55:35.904473 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 06:55:35.904829 (+ 356us) client_negotiation.cc:769] Sending connection context
0812 06:55:35.905070 (+ 241us) client_negotiation.cc:240] Negotiation successful
0812 06:55:35.905359 (+ 289us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":576,"thread_start_us":136,"threads_started":1}
I20260812 06:55:35.907749 29429 heartbeater.cc:347] Connected to a master server at 127.19.52.126:44739
I20260812 06:55:35.908010 29429 heartbeater.cc:464] Registering TS with master...
I20260812 06:55:35.908533 29429 heartbeater.cc:511] Master 127.19.52.126:44739 requested a full tablet report, sending...
I20260812 06:55:35.910112 29219 ts_manager.cc:194] Registered new tserver with Master: bc423acff98d45b79d8132e2a326d4a3 (127.19.52.65:34467)
WARNING: no policy specified for kudu/127.19.52.66@KRBTEST.COM; defaulting to no policy
I20260812 06:55:35.911307 29219 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.52.65@KRBTEST.COM'} at 127.19.52.65:35497
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.52.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.52.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517687384214-19665-0/krb5kdc/kudu_127.19.52.66.keytab.
Entry for principal kudu/127.19.52.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517687384214-19665-0/krb5kdc/kudu_127.19.52.66.keytab.
I20260812 06:55:35.932358 19665 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu
/tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517687384214-19665-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517687384214-19665-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517687384214-19665-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517687384214-19665-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.52.66:0
--local_ip_for_outbound_sockets=127.19.52.66
--webserver_interface=127.19.52.66
--webserver_port=0
--tserver_master_addrs=127.19.52.126:44739
--keytab_file=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517687384214-19665-0/krb5kdc/kudu_127.19.52.66.keytab
--principal=kudu/127.19.52.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.52.84:43735
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517687384214-19665-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517687384214-19665-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517687384214-19665-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 06:55:36.051102 29434 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:55:36.051457 29434 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:55:36.051540 29434 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 06:55:36.055823 29434 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 06:55:36.055943 29434 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 06:55:36.056046 29434 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.52.66
I20260812 06:55:36.062038 29434 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.52.84:43735
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517687384214-19665-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517687384214-19665-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517687384214-19665-0/krb5kdc/kudu_127.19.52.66.keytab
--principal=kudu/127.19.52.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.52.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517687384214-19665-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.19.52.66
--webserver_port=0
--tserver_master_addrs=127.19.52.126:44739
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29434
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.52.66
--log_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517687384214-19665-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 06:15:54 UTC on ccycloud.quasar-rnfzce.root.comops.site
build id 4581989
I20260812 06:55:36.063321 29434 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:55:36.064455 29434 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 06:55:36.074931 29443 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:55:36.075002 29440 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:55:36.075314 29434 server_base.cc:1034] running on GCE node
W20260812 06:55:36.074966 29441 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:55:36.076198 29434 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:55:36.077127 29434 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:55:36.078334 29434 hybrid_clock.cc:648] HybridClock initialized: now 1786517736078313 us; error 35 us; skew 500 ppm
Aug 12 06:55:36 dist-test-slave-65mx krb5kdc[29165](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517736, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.52.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 06:55:36.081418 29434 init.cc:377] Logged in from keytab as kudu/127.19.52.66@KRBTEST.COM (short username kudu)
I20260812 06:55:36.082432 29434 webserver.cc:460] Webserver started at http://127.19.52.66:46491/ using document root <none> and password file <none>
I20260812 06:55:36.083068 29434 fs_manager.cc:359] Metadata directory not provided
I20260812 06:55:36.083175 29434 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:55:36.083441 29434 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 06:55:36.085826 29434 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517687384214-19665-0/minicluster-data/ts-1/data/instance:
uuid: "53bcf3992d6f43a5874a0b70c1092229"
format_stamp: "Formatted at 2026-08-12 06:55:36 on dist-test-slave-65mx"
server_key: "f81fd25bc15d14e5b968b39f01f53ab6"
server_key_iv: "093b184705f6f34a39d15020bbc8cb43"
server_key_version: "encryptionkey@0"
I20260812 06:55:36.086462 29434 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517687384214-19665-0/minicluster-data/ts-1/wal/instance:
uuid: "53bcf3992d6f43a5874a0b70c1092229"
format_stamp: "Formatted at 2026-08-12 06:55:36 on dist-test-slave-65mx"
server_key: "f81fd25bc15d14e5b968b39f01f53ab6"
server_key_iv: "093b184705f6f34a39d15020bbc8cb43"
server_key_version: "encryptionkey@0"
I20260812 06:55:36.090823 29434 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260812 06:55:36.093652 29451 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 06:55:36.095050 29434 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 06:55:36.095221 29434 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517687384214-19665-0/minicluster-data/ts-1/data,/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517687384214-19665-0/minicluster-data/ts-1/wal
uuid: "53bcf3992d6f43a5874a0b70c1092229"
format_stamp: "Formatted at 2026-08-12 06:55:36 on dist-test-slave-65mx"
server_key: "f81fd25bc15d14e5b968b39f01f53ab6"
server_key_iv: "093b184705f6f34a39d15020bbc8cb43"
server_key_version: "encryptionkey@0"
I20260812 06:55:36.095358 29434 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517687384214-19665-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517687384214-19665-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517687384214-19665-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:55:36.111471 29434 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 06:55:36.114917 29434 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 06:55:36.115156 29434 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:55:36.115804 29434 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 06:55:36.116793 29434 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 06:55:36.116869 29434 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 06:55:36.116982 29434 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 06:55:36.117040 29434 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 06:55:36.127628 29434 rpc_server.cc:307] RPC server started. Bound to: 127.19.52.66:35811
I20260812 06:55:36.127653 29573 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.52.66:35811 every 8 connection(s)
I20260812 06:55:36.128734 29434 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517687384214-19665-0/minicluster-data/ts-1/data/info.pb
I20260812 06:55:36.130846 19665 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu as pid 29434
I20260812 06:55:36.130998 19665 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517687384214-19665-0/minicluster-data/ts-1/wal/instance
I20260812 06:55:36.131299 19665 external_mini_cluster.cc:1428] Setting key d235f871eb773ecf934299b52bdf109c
Aug 12 06:55:36 dist-test-slave-65mx krb5kdc[29165](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517736, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.52.66@KRBTEST.COM for kudu/127.19.52.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.19.52.67@KRBTEST.COM; defaulting to no policy
I20260812 06:55:36.145823 29259 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:36.130827 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.52.66:46389 (local address 127.19.52.126:44739)
0812 06:55:36.131041 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0812 06:55:36.131046 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 06:55:36.132306 (+ 1260us) server_negotiation.cc:407] Connection header received
0812 06:55:36.133337 (+ 1031us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:55:36.133343 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:55:36.133406 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:55:36.133496 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:55:36.135233 (+ 1737us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:36.136118 (+ 885us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:36.137779 (+ 1661us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:36.138192 (+ 413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:36.138341 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:55:36.141362 (+ 3021us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:55:36.141366 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:55:36.141370 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:55:36.141402 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:55:36.143345 (+ 1943us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:55:36.143987 (+ 642us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:55:36.143990 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:55:36.143991 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:55:36.144095 (+ 104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:55:36.144495 (+ 400us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:55:36.144498 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:55:36.144499 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:55:36.144766 (+ 267us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:55:36.144881 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0812 06:55:36.145424 (+ 543us) server_negotiation.cc:299] Negotiation successful
0812 06:55:36.145611 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":86}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.52.67@KRBTEST.COM" created.
I20260812 06:55:36.146729 29576 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:36.131318 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.126:44739 (local address 127.19.52.66:46389)
0812 06:55:36.132129 (+ 811us) negotiation.cc:107] Waiting for socket to connect
0812 06:55:36.132190 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0812 06:55:36.133124 (+ 934us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:55:36.133725 (+ 601us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:55:36.133734 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:55:36.134294 (+ 560us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:55:36.135039 (+ 745us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:55:36.135064 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:55:36.136299 (+ 1235us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:55:36.136306 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:55:36.137642 (+ 1336us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:55:36.137653 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:55:36.138371 (+ 718us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:55:36.138375 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:55:36.138552 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:55:36.139285 (+ 733us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 06:55:36.139303 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:55:36.141155 (+ 1852us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 06:55:36.143534 (+ 2379us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:55:36.143540 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:55:36.143560 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:55:36.143836 (+ 276us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:55:36.144186 (+ 350us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:55:36.144188 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:55:36.144190 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:55:36.144372 (+ 182us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:55:36.144881 (+ 509us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 06:55:36.144889 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 06:55:36.145167 (+ 278us) client_negotiation.cc:769] Sending connection context
0812 06:55:36.145365 (+ 198us) client_negotiation.cc:240] Negotiation successful
0812 06:55:36.145740 (+ 375us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":563,"thread_start_us":145,"threads_started":1}
I20260812 06:55:36.148070 29574 heartbeater.cc:347] Connected to a master server at 127.19.52.126:44739
I20260812 06:55:36.148339 29574 heartbeater.cc:464] Registering TS with master...
I20260812 06:55:36.149024 29574 heartbeater.cc:511] Master 127.19.52.126:44739 requested a full tablet report, sending...
I20260812 06:55:36.150290 29219 ts_manager.cc:194] Registered new tserver with Master: 53bcf3992d6f43a5874a0b70c1092229 (127.19.52.66:35811)
I20260812 06:55:36.151203 29219 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.52.66@KRBTEST.COM'} at 127.19.52.66:46389
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.52.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517687384214-19665-0/krb5kdc/kudu_127.19.52.67.keytab.
Entry for principal kudu/127.19.52.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517687384214-19665-0/krb5kdc/kudu_127.19.52.67.keytab.
I20260812 06:55:36.168833 19665 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu
/tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517687384214-19665-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517687384214-19665-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517687384214-19665-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517687384214-19665-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.52.67:0
--local_ip_for_outbound_sockets=127.19.52.67
--webserver_interface=127.19.52.67
--webserver_port=0
--tserver_master_addrs=127.19.52.126:44739
--keytab_file=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517687384214-19665-0/krb5kdc/kudu_127.19.52.67.keytab
--principal=kudu/127.19.52.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.52.84:43735
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517687384214-19665-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517687384214-19665-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517687384214-19665-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 06:55:36.288111 29579 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:55:36.288396 29579 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:55:36.288441 29579 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 06:55:36.292698 29579 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 06:55:36.292773 29579 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 06:55:36.292873 29579 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.52.67
I20260812 06:55:36.298486 29579 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.52.84:43735
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517687384214-19665-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517687384214-19665-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517687384214-19665-0/krb5kdc/kudu_127.19.52.67.keytab
--principal=kudu/127.19.52.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.52.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517687384214-19665-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.19.52.67
--webserver_port=0
--tserver_master_addrs=127.19.52.126:44739
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29579
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.52.67
--log_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517687384214-19665-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 06:15:54 UTC on ccycloud.quasar-rnfzce.root.comops.site
build id 4581989
I20260812 06:55:36.299728 29579 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:55:36.300915 29579 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 06:55:36.310932 29588 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:55:36.310999 29586 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:55:36.310945 29584 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:55:36.311347 29579 server_base.cc:1034] running on GCE node
I20260812 06:55:36.311926 29579 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:55:36.312763 29579 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:55:36.313949 29579 hybrid_clock.cc:648] HybridClock initialized: now 1786517736313928 us; error 35 us; skew 500 ppm
Aug 12 06:55:36 dist-test-slave-65mx krb5kdc[29165](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517736, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.52.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 06:55:36.317133 29579 init.cc:377] Logged in from keytab as kudu/127.19.52.67@KRBTEST.COM (short username kudu)
I20260812 06:55:36.318109 29579 webserver.cc:460] Webserver started at http://127.19.52.67:37325/ using document root <none> and password file <none>
I20260812 06:55:36.318706 29579 fs_manager.cc:359] Metadata directory not provided
I20260812 06:55:36.318771 29579 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:55:36.318998 29579 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 06:55:36.321174 29579 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517687384214-19665-0/minicluster-data/ts-2/data/instance:
uuid: "287c40803a7742f2b3f9a135b3a22a03"
format_stamp: "Formatted at 2026-08-12 06:55:36 on dist-test-slave-65mx"
server_key: "1cb0a8fbb304d64f7f57980e2806d0ac"
server_key_iv: "8c3a7cfcc69d58f36c1d9819cd70495d"
server_key_version: "encryptionkey@0"
I20260812 06:55:36.321772 29579 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517687384214-19665-0/minicluster-data/ts-2/wal/instance:
uuid: "287c40803a7742f2b3f9a135b3a22a03"
format_stamp: "Formatted at 2026-08-12 06:55:36 on dist-test-slave-65mx"
server_key: "1cb0a8fbb304d64f7f57980e2806d0ac"
server_key_iv: "8c3a7cfcc69d58f36c1d9819cd70495d"
server_key_version: "encryptionkey@0"
I20260812 06:55:36.326093 29579 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260812 06:55:36.328759 29595 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 06:55:36.330068 29579 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260812 06:55:36.330229 29579 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517687384214-19665-0/minicluster-data/ts-2/data,/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517687384214-19665-0/minicluster-data/ts-2/wal
uuid: "287c40803a7742f2b3f9a135b3a22a03"
format_stamp: "Formatted at 2026-08-12 06:55:36 on dist-test-slave-65mx"
server_key: "1cb0a8fbb304d64f7f57980e2806d0ac"
server_key_iv: "8c3a7cfcc69d58f36c1d9819cd70495d"
server_key_version: "encryptionkey@0"
I20260812 06:55:36.330363 29579 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517687384214-19665-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517687384214-19665-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517687384214-19665-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:55:36.350906 29579 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 06:55:36.355165 29579 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 06:55:36.355432 29579 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:55:36.356125 29579 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 06:55:36.357162 29579 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 06:55:36.357213 29579 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 06:55:36.357286 29579 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 06:55:36.357326 29579 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 06:55:36.367471 29579 rpc_server.cc:307] RPC server started. Bound to: 127.19.52.67:41389
I20260812 06:55:36.367517 29728 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.52.67:41389 every 8 connection(s)
I20260812 06:55:36.368706 29579 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517687384214-19665-0/minicluster-data/ts-2/data/info.pb
I20260812 06:55:36.377583 19665 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu as pid 29579
I20260812 06:55:36.377767 19665 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517687384214-19665-0/minicluster-data/ts-2/wal/instance
I20260812 06:55:36.378078 19665 external_mini_cluster.cc:1428] Setting key 369a82d1992efc65557db224022cfa86
Aug 12 06:55:36 dist-test-slave-65mx krb5kdc[29165](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517736, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.52.67@KRBTEST.COM for kudu/127.19.52.126@KRBTEST.COM
I20260812 06:55:36.385102 29259 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:36.371044 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.52.67:40769 (local address 127.19.52.126:44739)
0812 06:55:36.371225 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0812 06:55:36.371229 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 06:55:36.372130 (+ 901us) server_negotiation.cc:407] Connection header received
0812 06:55:36.373605 (+ 1475us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:55:36.373609 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:55:36.373673 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:55:36.373788 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:55:36.375490 (+ 1702us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:36.376104 (+ 614us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:36.377401 (+ 1297us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:36.377624 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:36.377734 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:55:36.381054 (+ 3320us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:55:36.381057 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:55:36.381060 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:55:36.381086 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:55:36.382711 (+ 1625us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:55:36.383338 (+ 627us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:55:36.383341 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:55:36.383342 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:55:36.383423 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:55:36.383883 (+ 460us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:55:36.383886 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:55:36.383887 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:55:36.384095 (+ 208us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:55:36.384182 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0812 06:55:36.384851 (+ 669us) server_negotiation.cc:299] Negotiation successful
0812 06:55:36.384971 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
I20260812 06:55:36.386053 29731 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:36.371394 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.126:44739 (local address 127.19.52.67:40769)
0812 06:55:36.371926 (+ 532us) negotiation.cc:107] Waiting for socket to connect
0812 06:55:36.371987 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0812 06:55:36.373344 (+ 1357us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:55:36.374061 (+ 717us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:55:36.374074 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:55:36.374649 (+ 575us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:55:36.375315 (+ 666us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:55:36.375330 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:55:36.376239 (+ 909us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:55:36.376243 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:55:36.377257 (+ 1014us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:55:36.377267 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:55:36.377799 (+ 532us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:55:36.377804 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:55:36.377981 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:55:36.378714 (+ 733us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 06:55:36.378732 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:55:36.380838 (+ 2106us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 06:55:36.382847 (+ 2009us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:55:36.382853 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:55:36.382868 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:55:36.383222 (+ 354us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:55:36.383538 (+ 316us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:55:36.383540 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:55:36.383542 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:55:36.383733 (+ 191us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:55:36.384215 (+ 482us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 06:55:36.384221 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 06:55:36.384489 (+ 268us) client_negotiation.cc:769] Sending connection context
0812 06:55:36.384725 (+ 236us) client_negotiation.cc:240] Negotiation successful
0812 06:55:36.385066 (+ 341us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":326,"thread_start_us":137,"threads_started":1}
I20260812 06:55:36.387367 29729 heartbeater.cc:347] Connected to a master server at 127.19.52.126:44739
I20260812 06:55:36.387681 29729 heartbeater.cc:464] Registering TS with master...
I20260812 06:55:36.388232 29729 heartbeater.cc:511] Master 127.19.52.126:44739 requested a full tablet report, sending...
I20260812 06:55:36.389649 29219 ts_manager.cc:194] Registered new tserver with Master: 287c40803a7742f2b3f9a135b3a22a03 (127.19.52.67:41389)
I20260812 06:55:36.390237 29219 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.52.67@KRBTEST.COM'} at 127.19.52.67:40769
I20260812 06:55:36.393484 19665 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.19.52.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.19.52.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.19.52.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517687384214-19665-0/krb5kdc/oryx_127.19.52.65.keytab.
Entry for principal oryx/127.19.52.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517687384214-19665-0/krb5kdc/oryx_127.19.52.65.keytab.
WARNING: no policy specified for oryx/127.19.52.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.19.52.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.19.52.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517687384214-19665-0/krb5kdc/oryx_127.19.52.66.keytab.
Entry for principal oryx/127.19.52.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517687384214-19665-0/krb5kdc/oryx_127.19.52.66.keytab.
WARNING: no policy specified for oryx/127.19.52.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.19.52.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.19.52.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517687384214-19665-0/krb5kdc/oryx_127.19.52.67.keytab.
Entry for principal oryx/127.19.52.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517687384214-19665-0/krb5kdc/oryx_127.19.52.67.keytab.
I20260812 06:55:36.485041 19665 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu with pid 29283
I20260812 06:55:36.496129 19665 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu with pid 29434
I20260812 06:55:36.504748 19665 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu with pid 29579
I20260812 06:55:36.514848 19665 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu with pid 29180
I20260812 06:55:36.524461 19665 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu
/tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517687384214-19665-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517687384214-19665-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517687384214-19665-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517687384214-19665-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.52.126:44739
--webserver_interface=127.19.52.126
--webserver_port=34635
--keytab_file=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517687384214-19665-0/krb5kdc/kudu_127.19.52.126.keytab
--principal=kudu/127.19.52.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.52.84:43735
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.19.52.126:44739
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517687384214-19665-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517687384214-19665-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517687384214-19665-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 06:55:36.647837 29738 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:55:36.648206 29738 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:55:36.648280 29738 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 06:55:36.652670 29738 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 06:55:36.652782 29738 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 06:55:36.652817 29738 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 06:55:36.652860 29738 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 06:55:36.652896 29738 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 06:55:36.658730 29738 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.52.84:43735
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517687384214-19665-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517687384214-19665-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.19.52.126:44739
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517687384214-19665-0/krb5kdc/kudu_127.19.52.126.keytab
--principal=kudu/127.19.52.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.52.126:44739
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517687384214-19665-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.19.52.126
--webserver_port=34635
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29738
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517687384214-19665-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 06:15:54 UTC on ccycloud.quasar-rnfzce.root.comops.site
build id 4581989
I20260812 06:55:36.660228 29738 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:55:36.662565 29738 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 06:55:36.673018 29748 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:55:36.673120 29746 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:55:36.673218 29738 server_base.cc:1034] running on GCE node
W20260812 06:55:36.673372 29745 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:55:36.674023 29738 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:55:36.675382 29738 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:55:36.676674 29738 hybrid_clock.cc:648] HybridClock initialized: now 1786517736676631 us; error 53 us; skew 500 ppm
Aug 12 06:55:36 dist-test-slave-65mx krb5kdc[29165](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517736, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.52.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 06:55:36.680006 29738 init.cc:377] Logged in from keytab as kudu/127.19.52.126@KRBTEST.COM (short username kudu)
I20260812 06:55:36.681183 29738 webserver.cc:460] Webserver started at http://127.19.52.126:34635/ using document root <none> and password file <none>
I20260812 06:55:36.681825 29738 fs_manager.cc:359] Metadata directory not provided
I20260812 06:55:36.681953 29738 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:55:36.686679 29738 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260812 06:55:36.689345 29756 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 06:55:36.690883 29738 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 06:55:36.691185 29738 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517687384214-19665-0/minicluster-data/master-0/data,/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517687384214-19665-0/minicluster-data/master-0/wal
uuid: "93edba2c8aca4bee8e1e0869d33fdb59"
format_stamp: "Formatted at 2026-08-12 06:55:35 on dist-test-slave-65mx"
server_key: "c142b935a9cb1ef8912af34cb9397ade"
server_key_iv: "cc33419d8afc8243f7e0b8cc7f0c2da3"
server_key_version: "encryptionkey@0"
I20260812 06:55:36.691789 29738 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517687384214-19665-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517687384214-19665-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517687384214-19665-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:55:36.703888 29738 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 06:55:36.708431 29738 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 06:55:36.708693 29738 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:55:36.717584 29738 rpc_server.cc:307] RPC server started. Bound to: 127.19.52.126:44739
I20260812 06:55:36.717619 29816 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.52.126:44739 every 8 connection(s)
I20260812 06:55:36.719033 29738 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517687384214-19665-0/minicluster-data/master-0/data/info.pb
I20260812 06:55:36.723109 19665 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu as pid 29738
I20260812 06:55:36.723661 19665 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu
/tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517687384214-19665-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517687384214-19665-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517687384214-19665-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517687384214-19665-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.52.65:34467
--local_ip_for_outbound_sockets=127.19.52.65
--tserver_master_addrs=127.19.52.126:44739
--webserver_port=44731
--webserver_interface=127.19.52.65
--keytab_file=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517687384214-19665-0/krb5kdc/kudu_127.19.52.65.keytab
--principal=kudu/127.19.52.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.52.84:43735
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.19.52.65
--keytab_file=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517687384214-19665-0/krb5kdc/oryx_127.19.52.65.keytab with env {KRB5CCNAME=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517687384214-19665-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517687384214-19665-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517687384214-19665-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 06:55:36.724382 29817 sys_catalog.cc:263] Verifying existing consensus state
I20260812 06:55:36.726945 29817 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 93edba2c8aca4bee8e1e0869d33fdb59: Bootstrap starting.
I20260812 06:55:36.746692 29817 log.cc:824] T 00000000000000000000000000000000 P 93edba2c8aca4bee8e1e0869d33fdb59: Log is configured to *not* fsync() on all Append() calls
I20260812 06:55:36.754599 29817 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 93edba2c8aca4bee8e1e0869d33fdb59: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260812 06:55:36.755077 29817 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 93edba2c8aca4bee8e1e0869d33fdb59: Bootstrap complete.
I20260812 06:55:36.760332 29817 raft_consensus.cc:348] T 00000000000000000000000000000000 P 93edba2c8aca4bee8e1e0869d33fdb59 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "93edba2c8aca4bee8e1e0869d33fdb59" member_type: VOTER last_known_addr { host: "127.19.52.126" port: 44739 } }
I20260812 06:55:36.760874 29817 raft_consensus.cc:729] T 00000000000000000000000000000000 P 93edba2c8aca4bee8e1e0869d33fdb59 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 93edba2c8aca4bee8e1e0869d33fdb59, State: Initialized, Role: FOLLOWER
I20260812 06:55:36.761425 29817 consensus_queue.cc:260] T 00000000000000000000000000000000 P 93edba2c8aca4bee8e1e0869d33fdb59 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "93edba2c8aca4bee8e1e0869d33fdb59" member_type: VOTER last_known_addr { host: "127.19.52.126" port: 44739 } }
I20260812 06:55:36.761548 29817 raft_consensus.cc:388] T 00000000000000000000000000000000 P 93edba2c8aca4bee8e1e0869d33fdb59 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 06:55:36.761642 29817 raft_consensus.cc:482] T 00000000000000000000000000000000 P 93edba2c8aca4bee8e1e0869d33fdb59 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 06:55:36.761744 29817 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 93edba2c8aca4bee8e1e0869d33fdb59 [term 1 FOLLOWER]: Advancing to term 2
I20260812 06:55:36.763942 29817 raft_consensus.cc:504] T 00000000000000000000000000000000 P 93edba2c8aca4bee8e1e0869d33fdb59 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "93edba2c8aca4bee8e1e0869d33fdb59" member_type: VOTER last_known_addr { host: "127.19.52.126" port: 44739 } }
I20260812 06:55:36.764356 29817 leader_election.cc:302] T 00000000000000000000000000000000 P 93edba2c8aca4bee8e1e0869d33fdb59 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 93edba2c8aca4bee8e1e0869d33fdb59; no voters:
I20260812 06:55:36.764976 29817 leader_election.cc:288] T 00000000000000000000000000000000 P 93edba2c8aca4bee8e1e0869d33fdb59 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260812 06:55:36.765080 29822 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 93edba2c8aca4bee8e1e0869d33fdb59 [term 2 FOLLOWER]: Leader election won for term 2
I20260812 06:55:36.765280 29822 raft_consensus.cc:686] T 00000000000000000000000000000000 P 93edba2c8aca4bee8e1e0869d33fdb59 [term 2 LEADER]: Becoming Leader. State: Replica: 93edba2c8aca4bee8e1e0869d33fdb59, State: Running, Role: LEADER
I20260812 06:55:36.765599 29822 consensus_queue.cc:237] T 00000000000000000000000000000000 P 93edba2c8aca4bee8e1e0869d33fdb59 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "93edba2c8aca4bee8e1e0869d33fdb59" member_type: VOTER last_known_addr { host: "127.19.52.126" port: 44739 } }
I20260812 06:55:36.766372 29817 sys_catalog.cc:564] T 00000000000000000000000000000000 P 93edba2c8aca4bee8e1e0869d33fdb59 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 06:55:36.766737 29822 sys_catalog.cc:455] T 00000000000000000000000000000000 P 93edba2c8aca4bee8e1e0869d33fdb59 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 93edba2c8aca4bee8e1e0869d33fdb59. Latest consensus state: current_term: 2 leader_uuid: "93edba2c8aca4bee8e1e0869d33fdb59" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "93edba2c8aca4bee8e1e0869d33fdb59" member_type: VOTER last_known_addr { host: "127.19.52.126" port: 44739 } } }
I20260812 06:55:36.766870 29822 sys_catalog.cc:458] T 00000000000000000000000000000000 P 93edba2c8aca4bee8e1e0869d33fdb59 [sys.catalog]: This master's current role is: LEADER
I20260812 06:55:36.767297 29826 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 06:55:36.767310 29823 sys_catalog.cc:455] T 00000000000000000000000000000000 P 93edba2c8aca4bee8e1e0869d33fdb59 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "93edba2c8aca4bee8e1e0869d33fdb59" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "93edba2c8aca4bee8e1e0869d33fdb59" member_type: VOTER last_known_addr { host: "127.19.52.126" port: 44739 } } }
I20260812 06:55:36.767620 29823 sys_catalog.cc:458] T 00000000000000000000000000000000 P 93edba2c8aca4bee8e1e0869d33fdb59 [sys.catalog]: This master's current role is: LEADER
I20260812 06:55:36.771569 29826 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 06:55:36.772301 29826 catalog_manager.cc:1259] Loaded cluster ID: 683b7c7b28f6489797e43d30d96f8a0d
I20260812 06:55:36.772370 29826 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 06:55:36.775637 29826 catalog_manager.cc:1504] Loading token signing keys...
I20260812 06:55:36.776779 29826 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 93edba2c8aca4bee8e1e0869d33fdb59: Loaded TSK: 0
I20260812 06:55:36.778723 29826 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260812 06:55:36.854255 29819 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:55:36.854746 29819 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:55:36.854882 29819 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 06:55:36.859483 29819 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 06:55:36.859627 29819 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 06:55:36.859740 29819 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.52.65
I20260812 06:55:36.865818 29819 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.52.84:43735
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517687384214-19665-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517687384214-19665-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517687384214-19665-0/krb5kdc/oryx_127.19.52.65.keytab
--principal=oryx/127.19.52.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.52.65:34467
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517687384214-19665-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.19.52.65
--webserver_port=44731
--tserver_master_addrs=127.19.52.126:44739
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29819
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.52.65
--log_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517687384214-19665-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 06:15:54 UTC on ccycloud.quasar-rnfzce.root.comops.site
build id 4581989
I20260812 06:55:36.867235 29819 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:55:36.868685 29819 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 06:55:36.880370 29849 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:55:36.880460 29847 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:55:36.880399 29846 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:55:36.881147 29819 server_base.cc:1034] running on GCE node
I20260812 06:55:36.881701 29819 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:55:36.882440 29819 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:55:36.883634 29819 hybrid_clock.cc:648] HybridClock initialized: now 1786517736883603 us; error 55 us; skew 500 ppm
Aug 12 06:55:36 dist-test-slave-65mx krb5kdc[29165](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517736, etypes {rep=17 tkt=17 ses=17}, oryx/127.19.52.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 06:55:36.886644 29819 init.cc:377] Logged in from keytab as oryx/127.19.52.65@KRBTEST.COM (short username oryx)
I20260812 06:55:36.887745 29819 webserver.cc:460] Webserver started at http://127.19.52.65:44731/ using document root <none> and password file <none>
I20260812 06:55:36.888352 29819 fs_manager.cc:359] Metadata directory not provided
I20260812 06:55:36.888437 29819 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:55:36.892925 29819 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.000s
I20260812 06:55:36.895251 29857 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 06:55:36.896790 29819 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 06:55:36.896932 29819 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517687384214-19665-0/minicluster-data/ts-0/data,/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517687384214-19665-0/minicluster-data/ts-0/wal
uuid: "bc423acff98d45b79d8132e2a326d4a3"
format_stamp: "Formatted at 2026-08-12 06:55:35 on dist-test-slave-65mx"
server_key: "bef05a5cc98d6121a2f0a05d668ffece"
server_key_iv: "43745e12c92ea977af7360a2ecbd8517"
server_key_version: "encryptionkey@0"
I20260812 06:55:36.897539 29819 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517687384214-19665-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517687384214-19665-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517687384214-19665-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:55:36.912425 29819 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 06:55:36.915920 29819 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 06:55:36.916148 29819 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:55:36.916929 29819 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 06:55:36.917855 29819 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 06:55:36.917906 29819 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 06:55:36.917977 29819 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 06:55:36.918007 29819 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 06:55:36.928181 29819 rpc_server.cc:307] RPC server started. Bound to: 127.19.52.65:34467
I20260812 06:55:36.928227 29986 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.52.65:34467 every 8 connection(s)
I20260812 06:55:36.929327 29819 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517687384214-19665-0/minicluster-data/ts-0/data/info.pb
I20260812 06:55:36.932667 19665 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu as pid 29819
I20260812 06:55:36.933147 19665 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu
/tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517687384214-19665-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517687384214-19665-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517687384214-19665-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517687384214-19665-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.52.66:35811
--local_ip_for_outbound_sockets=127.19.52.66
--tserver_master_addrs=127.19.52.126:44739
--webserver_port=46491
--webserver_interface=127.19.52.66
--keytab_file=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517687384214-19665-0/krb5kdc/kudu_127.19.52.66.keytab
--principal=kudu/127.19.52.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.52.84:43735
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.19.52.66
--keytab_file=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517687384214-19665-0/krb5kdc/oryx_127.19.52.66.keytab with env {KRB5CCNAME=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517687384214-19665-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517687384214-19665-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517687384214-19665-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 12 06:55:36 dist-test-slave-65mx krb5kdc[29165](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.52.65@KRBTEST.COM for oryx/127.19.52.126@KRBTEST.COM, Server not found in Kerberos database
Aug 12 06:55:36 dist-test-slave-65mx krb5kdc[29165](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.52.65@KRBTEST.COM for oryx/127.19.52.126@KRBTEST.COM, Server not found in Kerberos database
W20260812 06:55:36.947592 29990 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 06:55:36.931864 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.52.65:53827 (local address 127.19.52.126:44739)
0812 06:55:36.934980 (+ 3116us) server_negotiation.cc:206] Beginning negotiation
0812 06:55:36.935000 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0812 06:55:36.935041 (+ 41us) server_negotiation.cc:407] Connection header received
0812 06:55:36.935261 (+ 220us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:55:36.935271 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:55:36.935601 (+ 330us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:55:36.935907 (+ 306us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:55:36.937809 (+ 1902us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:36.938979 (+ 1170us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:36.941201 (+ 2222us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:36.941749 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:36.941898 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:55:36.946782 (+ 4884us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.19.52.65:53827: BlockingRecv error: recv got EOF from 127.19.52.65:53827 (error 108)
Metrics: {"server-negotiator.queue_time_us":350,"thread_start_us":164,"threads_started":1}
I20260812 06:55:36.947602 29991 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:36.932032 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.126:44739 (local address 127.19.52.65:53827)
0812 06:55:36.932884 (+ 852us) negotiation.cc:107] Waiting for socket to connect
0812 06:55:36.932945 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0812 06:55:36.933966 (+ 1021us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:55:36.936126 (+ 2160us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:55:36.936140 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:55:36.936792 (+ 652us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:55:36.937579 (+ 787us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:55:36.937606 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:55:36.939156 (+ 1550us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:55:36.939162 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:55:36.941016 (+ 1854us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:55:36.941045 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:55:36.941963 (+ 918us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:55:36.941971 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:55:36.942265 (+ 294us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:55:36.943011 (+ 746us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 06:55:36.943055 (+ 44us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:55:36.946532 (+ 3477us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.19.52.126:44739: Server oryx/127.19.52.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":623,"thread_start_us":150,"threads_started":1}
W20260812 06:55:36.948382 29987 heartbeater.cc:644] Failed to heartbeat to 127.19.52.126:44739 (0 consecutive failures): Not authorized: Failed to ping master at 127.19.52.126:44739: Client connection negotiation failed: client connection to 127.19.52.126:44739: Server oryx/127.19.52.126@KRBTEST.COM not found in Kerberos database
Aug 12 06:55:36 dist-test-slave-65mx krb5kdc[29165](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.52.65@KRBTEST.COM for oryx/127.19.52.126@KRBTEST.COM, Server not found in Kerberos database
Aug 12 06:55:36 dist-test-slave-65mx krb5kdc[29165](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.52.65@KRBTEST.COM for oryx/127.19.52.126@KRBTEST.COM, Server not found in Kerberos database
I20260812 06:55:36.955930 29991 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:36.948970 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.126:44739 (local address 127.19.52.65:57219)
0812 06:55:36.949149 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0812 06:55:36.949167 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0812 06:55:36.949262 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:55:36.949532 (+ 270us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:55:36.949535 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:55:36.949679 (+ 144us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:55:36.949786 (+ 107us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:55:36.949791 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:55:36.951107 (+ 1316us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:55:36.951111 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:55:36.951868 (+ 757us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:55:36.951881 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:55:36.952687 (+ 806us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:55:36.952690 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:55:36.952768 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:55:36.953392 (+ 624us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 06:55:36.953405 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:55:36.955790 (+ 2385us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.19.52.126:44739: Server oryx/127.19.52.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":91}
W20260812 06:55:36.956516 29990 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 06:55:36.949100 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.52.65:57219 (local address 127.19.52.126:44739)
0812 06:55:36.949229 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0812 06:55:36.949233 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 06:55:36.949253 (+ 20us) server_negotiation.cc:407] Connection header received
0812 06:55:36.949340 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:55:36.949343 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:55:36.949397 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:55:36.949500 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:55:36.949906 (+ 406us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:36.950882 (+ 976us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:36.952006 (+ 1124us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:36.952517 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:36.952676 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:55:36.956030 (+ 3354us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.19.52.65:57219: BlockingRecv error: recv got EOF from 127.19.52.65:57219 (error 108)
Metrics: {"server-negotiator.queue_time_us":39}
Aug 12 06:55:36 dist-test-slave-65mx krb5kdc[29165](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.52.65@KRBTEST.COM for oryx/127.19.52.126@KRBTEST.COM, Server not found in Kerberos database
Aug 12 06:55:36 dist-test-slave-65mx krb5kdc[29165](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.52.65@KRBTEST.COM for oryx/127.19.52.126@KRBTEST.COM, Server not found in Kerberos database
I20260812 06:55:36.963888 29991 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:36.957004 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.126:44739 (local address 127.19.52.65:57253)
0812 06:55:36.957136 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0812 06:55:36.957153 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0812 06:55:36.957269 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:55:36.957592 (+ 323us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:55:36.957598 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:55:36.957754 (+ 156us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:55:36.957847 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:55:36.957852 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:55:36.959023 (+ 1171us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:55:36.959025 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:55:36.959704 (+ 679us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:55:36.959717 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:55:36.960352 (+ 635us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:55:36.960356 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:55:36.960460 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:55:36.961200 (+ 740us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 06:55:36.961218 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:55:36.963670 (+ 2452us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.19.52.126:44739: Server oryx/127.19.52.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":59}
W20260812 06:55:36.964103 29990 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 06:55:36.956827 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.52.65:57253 (local address 127.19.52.126:44739)
0812 06:55:36.956964 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0812 06:55:36.956968 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 06:55:36.957235 (+ 267us) server_negotiation.cc:407] Connection header received
0812 06:55:36.957362 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:55:36.957366 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:55:36.957420 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:55:36.957513 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:55:36.957966 (+ 453us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:36.958813 (+ 847us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:36.959857 (+ 1044us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:36.960190 (+ 333us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:36.960310 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:55:36.963931 (+ 3621us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.19.52.65:57253: BlockingRecv error: recv got EOF from 127.19.52.65:57253 (error 108)
Metrics: {"server-negotiator.queue_time_us":33}
W20260812 06:55:36.964280 29987 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260812 06:55:37.063119 29992 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:55:37.063422 29992 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:55:37.063470 29992 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 06:55:37.067898 29992 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 06:55:37.067996 29992 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 06:55:37.068148 29992 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.52.66
I20260812 06:55:37.074347 29992 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.52.84:43735
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517687384214-19665-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517687384214-19665-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517687384214-19665-0/krb5kdc/oryx_127.19.52.66.keytab
--principal=oryx/127.19.52.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.52.66:35811
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517687384214-19665-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.19.52.66
--webserver_port=46491
--tserver_master_addrs=127.19.52.126:44739
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29992
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.52.66
--log_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517687384214-19665-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 06:15:54 UTC on ccycloud.quasar-rnfzce.root.comops.site
build id 4581989
I20260812 06:55:37.075778 29992 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:55:37.077026 29992 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 06:55:37.088716 29998 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:55:37.088938 29992 server_base.cc:1034] running on GCE node
W20260812 06:55:37.088735 30001 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:55:37.088723 29999 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:55:37.089519 29992 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:55:37.090188 29992 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:55:37.091434 29992 hybrid_clock.cc:648] HybridClock initialized: now 1786517737091402 us; error 42 us; skew 500 ppm
Aug 12 06:55:37 dist-test-slave-65mx krb5kdc[29165](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517737, etypes {rep=17 tkt=17 ses=17}, oryx/127.19.52.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 06:55:37.094764 29992 init.cc:377] Logged in from keytab as oryx/127.19.52.66@KRBTEST.COM (short username oryx)
I20260812 06:55:37.095862 29992 webserver.cc:460] Webserver started at http://127.19.52.66:46491/ using document root <none> and password file <none>
I20260812 06:55:37.096875 29992 fs_manager.cc:359] Metadata directory not provided
I20260812 06:55:37.096997 29992 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:55:37.101696 29992 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.000s
I20260812 06:55:37.104228 30010 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 06:55:37.105822 29992 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260812 06:55:37.105969 29992 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517687384214-19665-0/minicluster-data/ts-1/data,/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517687384214-19665-0/minicluster-data/ts-1/wal
uuid: "53bcf3992d6f43a5874a0b70c1092229"
format_stamp: "Formatted at 2026-08-12 06:55:36 on dist-test-slave-65mx"
server_key: "f81fd25bc15d14e5b968b39f01f53ab6"
server_key_iv: "093b184705f6f34a39d15020bbc8cb43"
server_key_version: "encryptionkey@0"
I20260812 06:55:37.106591 29992 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517687384214-19665-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517687384214-19665-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517687384214-19665-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:55:37.139707 29992 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 06:55:37.144061 29992 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 06:55:37.144240 29992 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:55:37.144918 29992 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 06:55:37.145767 29992 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 06:55:37.145816 29992 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 06:55:37.145859 29992 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 06:55:37.145874 29992 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 06:55:37.156363 29992 rpc_server.cc:307] RPC server started. Bound to: 127.19.52.66:35811
I20260812 06:55:37.156394 30139 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.52.66:35811 every 8 connection(s)
I20260812 06:55:37.157505 29992 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517687384214-19665-0/minicluster-data/ts-1/data/info.pb
I20260812 06:55:37.162884 19665 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu as pid 29992
I20260812 06:55:37.163420 19665 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu
/tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517687384214-19665-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517687384214-19665-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517687384214-19665-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517687384214-19665-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.52.67:41389
--local_ip_for_outbound_sockets=127.19.52.67
--tserver_master_addrs=127.19.52.126:44739
--webserver_port=37325
--webserver_interface=127.19.52.67
--keytab_file=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517687384214-19665-0/krb5kdc/kudu_127.19.52.67.keytab
--principal=kudu/127.19.52.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.52.84:43735
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.19.52.67
--keytab_file=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517687384214-19665-0/krb5kdc/oryx_127.19.52.67.keytab with env {KRB5CCNAME=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517687384214-19665-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517687384214-19665-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517687384214-19665-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 12 06:55:37 dist-test-slave-65mx krb5kdc[29165](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.52.66@KRBTEST.COM for oryx/127.19.52.126@KRBTEST.COM, Server not found in Kerberos database
Aug 12 06:55:37 dist-test-slave-65mx krb5kdc[29165](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.52.66@KRBTEST.COM for oryx/127.19.52.126@KRBTEST.COM, Server not found in Kerberos database
W20260812 06:55:37.172063 29990 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 06:55:37.159744 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.52.66:38571 (local address 127.19.52.126:44739)
0812 06:55:37.159928 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0812 06:55:37.159932 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 06:55:37.161244 (+ 1312us) server_negotiation.cc:407] Connection header received
0812 06:55:37.162209 (+ 965us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:55:37.162213 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:55:37.162284 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:55:37.162398 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:55:37.164015 (+ 1617us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:37.164763 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:37.166764 (+ 2001us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:37.166996 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:37.167116 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:55:37.171855 (+ 4739us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.19.52.66:38571: BlockingRecv error: recv got EOF from 127.19.52.66:38571 (error 108)
Metrics: {"server-negotiator.queue_time_us":83}
I20260812 06:55:37.172859 30142 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:37.160138 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.126:44739 (local address 127.19.52.66:38571)
0812 06:55:37.161065 (+ 927us) negotiation.cc:107] Waiting for socket to connect
0812 06:55:37.161110 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0812 06:55:37.161979 (+ 869us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:55:37.162747 (+ 768us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:55:37.162755 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:55:37.163236 (+ 481us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:55:37.163840 (+ 604us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:55:37.163856 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:55:37.164923 (+ 1067us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:55:37.164925 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:55:37.166592 (+ 1667us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:55:37.166604 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:55:37.167182 (+ 578us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:55:37.167186 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:55:37.167573 (+ 387us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:55:37.168418 (+ 845us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 06:55:37.168444 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:55:37.171650 (+ 3206us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.19.52.126:44739: Server oryx/127.19.52.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":716,"thread_start_us":133,"threads_started":1}
W20260812 06:55:37.173557 30140 heartbeater.cc:644] Failed to heartbeat to 127.19.52.126:44739 (0 consecutive failures): Not authorized: Failed to ping master at 127.19.52.126:44739: Client connection negotiation failed: client connection to 127.19.52.126:44739: Server oryx/127.19.52.126@KRBTEST.COM not found in Kerberos database
Aug 12 06:55:37 dist-test-slave-65mx krb5kdc[29165](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.52.66@KRBTEST.COM for oryx/127.19.52.126@KRBTEST.COM, Server not found in Kerberos database
Aug 12 06:55:37 dist-test-slave-65mx krb5kdc[29165](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.52.66@KRBTEST.COM for oryx/127.19.52.126@KRBTEST.COM, Server not found in Kerberos database
I20260812 06:55:37.182013 30142 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:37.174325 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.126:44739 (local address 127.19.52.66:54797)
0812 06:55:37.174481 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0812 06:55:37.174499 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0812 06:55:37.174589 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:55:37.174953 (+ 364us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:55:37.174955 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:55:37.175115 (+ 160us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:55:37.175223 (+ 108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:55:37.175229 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:55:37.176737 (+ 1508us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:55:37.176740 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:55:37.177358 (+ 618us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:55:37.177368 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:55:37.178139 (+ 771us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:55:37.178142 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:55:37.178231 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:55:37.179026 (+ 795us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 06:55:37.179039 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:55:37.181839 (+ 2800us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.19.52.126:44739: Server oryx/127.19.52.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":55}
W20260812 06:55:37.182271 29990 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 06:55:37.174440 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.52.66:54797 (local address 127.19.52.126:44739)
0812 06:55:37.174636 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0812 06:55:37.174643 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 06:55:37.174664 (+ 21us) server_negotiation.cc:407] Connection header received
0812 06:55:37.174727 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:55:37.174732 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:55:37.174796 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:55:37.174905 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:55:37.175362 (+ 457us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:37.176468 (+ 1106us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:37.177565 (+ 1097us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:37.177948 (+ 383us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:37.178090 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:55:37.182113 (+ 4023us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.19.52.66:54797: BlockingRecv error: recv got EOF from 127.19.52.66:54797 (error 108)
Metrics: {"server-negotiator.queue_time_us":99}
Aug 12 06:55:37 dist-test-slave-65mx krb5kdc[29165](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.52.66@KRBTEST.COM for oryx/127.19.52.126@KRBTEST.COM, Server not found in Kerberos database
Aug 12 06:55:37 dist-test-slave-65mx krb5kdc[29165](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.52.66@KRBTEST.COM for oryx/127.19.52.126@KRBTEST.COM, Server not found in Kerberos database
I20260812 06:55:37.188787 30142 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:37.183196 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.126:44739 (local address 127.19.52.66:54169)
0812 06:55:37.183329 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0812 06:55:37.183346 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0812 06:55:37.183457 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:55:37.183758 (+ 301us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:55:37.183761 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:55:37.183940 (+ 179us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:55:37.184030 (+ 90us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:55:37.184035 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:55:37.184860 (+ 825us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:55:37.184863 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:55:37.185398 (+ 535us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:55:37.185406 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:55:37.185869 (+ 463us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:55:37.185872 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:55:37.185956 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:55:37.186581 (+ 625us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 06:55:37.186596 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:55:37.188631 (+ 2035us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.19.52.126:44739: Server oryx/127.19.52.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":50}
W20260812 06:55:37.188961 29990 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 06:55:37.183269 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.52.66:54169 (local address 127.19.52.126:44739)
0812 06:55:37.183432 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0812 06:55:37.183436 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 06:55:37.183452 (+ 16us) server_negotiation.cc:407] Connection header received
0812 06:55:37.183607 (+ 155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:55:37.183610 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:55:37.183652 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:55:37.183738 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:55:37.184148 (+ 410us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:37.184748 (+ 600us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:37.185531 (+ 783us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:37.185750 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:37.185864 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:55:37.188790 (+ 2926us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.19.52.66:54169: BlockingRecv error: recv got EOF from 127.19.52.66:54169 (error 108)
Metrics: {"server-negotiator.queue_time_us":81}
W20260812 06:55:37.189061 30140 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260812 06:55:37.294955 30143 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:55:37.295260 30143 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:55:37.295308 30143 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 06:55:37.299749 30143 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 06:55:37.299834 30143 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 06:55:37.299897 30143 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.52.67
I20260812 06:55:37.305835 30143 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.52.84:43735
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517687384214-19665-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517687384214-19665-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517687384214-19665-0/krb5kdc/oryx_127.19.52.67.keytab
--principal=oryx/127.19.52.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.52.67:41389
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517687384214-19665-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.19.52.67
--webserver_port=37325
--tserver_master_addrs=127.19.52.126:44739
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30143
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.52.67
--log_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517687384214-19665-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 06:15:54 UTC on ccycloud.quasar-rnfzce.root.comops.site
build id 4581989
I20260812 06:55:37.307026 30143 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:55:37.308112 30143 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 06:55:37.318907 30148 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:55:37.318851 30151 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:55:37.319252 30143 server_base.cc:1034] running on GCE node
W20260812 06:55:37.318851 30149 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:55:37.319870 30143 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:55:37.320667 30143 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:55:37.321874 30143 hybrid_clock.cc:648] HybridClock initialized: now 1786517737321853 us; error 32 us; skew 500 ppm
Aug 12 06:55:37 dist-test-slave-65mx krb5kdc[29165](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517737, etypes {rep=17 tkt=17 ses=17}, oryx/127.19.52.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 06:55:37.325320 30143 init.cc:377] Logged in from keytab as oryx/127.19.52.67@KRBTEST.COM (short username oryx)
I20260812 06:55:37.326416 30143 webserver.cc:460] Webserver started at http://127.19.52.67:37325/ using document root <none> and password file <none>
I20260812 06:55:37.327028 30143 fs_manager.cc:359] Metadata directory not provided
I20260812 06:55:37.327118 30143 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:55:37.331558 30143 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.000s
I20260812 06:55:37.333894 30159 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 06:55:37.335355 30143 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260812 06:55:37.335494 30143 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517687384214-19665-0/minicluster-data/ts-2/data,/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517687384214-19665-0/minicluster-data/ts-2/wal
uuid: "287c40803a7742f2b3f9a135b3a22a03"
format_stamp: "Formatted at 2026-08-12 06:55:36 on dist-test-slave-65mx"
server_key: "1cb0a8fbb304d64f7f57980e2806d0ac"
server_key_iv: "8c3a7cfcc69d58f36c1d9819cd70495d"
server_key_version: "encryptionkey@0"
I20260812 06:55:37.336122 30143 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517687384214-19665-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517687384214-19665-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517687384214-19665-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:55:37.351311 30143 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 06:55:37.355684 30143 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 06:55:37.355937 30143 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:55:37.356851 30143 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 06:55:37.358217 30143 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 06:55:37.358273 30143 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 06:55:37.358350 30143 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 06:55:37.358389 30143 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 06:55:37.370430 30143 rpc_server.cc:307] RPC server started. Bound to: 127.19.52.67:41389
I20260812 06:55:37.370518 30284 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.52.67:41389 every 8 connection(s)
I20260812 06:55:37.371658 30143 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517687384214-19665-0/minicluster-data/ts-2/data/info.pb
I20260812 06:55:37.372449 19665 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu as pid 30143
Aug 12 06:55:37 dist-test-slave-65mx krb5kdc[29165](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.52.67@KRBTEST.COM for oryx/127.19.52.126@KRBTEST.COM, Server not found in Kerberos database
Aug 12 06:55:37 dist-test-slave-65mx krb5kdc[29165](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.52.67@KRBTEST.COM for oryx/127.19.52.126@KRBTEST.COM, Server not found in Kerberos database
I20260812 06:55:37.385025 29990 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:37.373367 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39396 (local address 127.19.52.126:44739)
0812 06:55:37.373544 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0812 06:55:37.373549 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 06:55:37.374215 (+ 666us) server_negotiation.cc:407] Connection header received
0812 06:55:37.374271 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:55:37.374274 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:55:37.374326 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:55:37.374418 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:55:37.375312 (+ 894us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:37.376159 (+ 847us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:37.377776 (+ 1617us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:37.378042 (+ 266us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:37.378169 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:55:37.380198 (+ 2029us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:55:37.380203 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:55:37.380219 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:55:37.380250 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:55:37.382488 (+ 2238us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:55:37.383038 (+ 550us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:55:37.383044 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:55:37.383049 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:55:37.383227 (+ 178us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:55:37.383590 (+ 363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:55:37.383595 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:55:37.383602 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:55:37.384028 (+ 426us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:55:37.384263 (+ 235us) server_negotiation.cc:1035] Waiting for connection context
0812 06:55:37.384538 (+ 275us) server_negotiation.cc:299] Negotiation successful
0812 06:55:37.384851 (+ 313us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
W20260812 06:55:37.385346 30291 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 06:55:37.374286 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.52.67:48473 (local address 127.19.52.126:44739)
0812 06:55:37.374625 (+ 339us) server_negotiation.cc:206] Beginning negotiation
0812 06:55:37.374629 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 06:55:37.375587 (+ 958us) server_negotiation.cc:407] Connection header received
0812 06:55:37.376453 (+ 866us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:55:37.376457 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:55:37.376516 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:55:37.376676 (+ 160us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:55:37.377947 (+ 1271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:37.378548 (+ 601us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:37.380467 (+ 1919us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:37.380834 (+ 367us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:37.381003 (+ 169us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:55:37.385233 (+ 4230us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.19.52.67:48473: BlockingRecv error: recv got EOF from 127.19.52.67:48473 (error 108)
Metrics: {"server-negotiator.queue_time_us":274,"thread_start_us":144,"threads_started":1}
I20260812 06:55:37.386103 30292 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:37.374510 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.126:44739 (local address 127.19.52.67:48473)
0812 06:55:37.375329 (+ 819us) negotiation.cc:107] Waiting for socket to connect
0812 06:55:37.375389 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0812 06:55:37.376221 (+ 832us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:55:37.376866 (+ 645us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:55:37.376874 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:55:37.377276 (+ 402us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:55:37.377788 (+ 512us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:55:37.377801 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:55:37.378708 (+ 907us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:55:37.378711 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:55:37.380325 (+ 1614us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:55:37.380336 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:55:37.381038 (+ 702us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:55:37.381042 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:55:37.381209 (+ 167us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:55:37.381943 (+ 734us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 06:55:37.381959 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:55:37.385058 (+ 3099us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.19.52.126:44739: Server oryx/127.19.52.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":597,"thread_start_us":166,"threads_started":1}
W20260812 06:55:37.386814 30286 heartbeater.cc:644] Failed to heartbeat to 127.19.52.126:44739 (0 consecutive failures): Not authorized: Failed to ping master at 127.19.52.126:44739: Client connection negotiation failed: client connection to 127.19.52.126:44739: Server oryx/127.19.52.126@KRBTEST.COM not found in Kerberos database
Aug 12 06:55:37 dist-test-slave-65mx krb5kdc[29165](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.52.67@KRBTEST.COM for oryx/127.19.52.126@KRBTEST.COM, Server not found in Kerberos database
Aug 12 06:55:37 dist-test-slave-65mx krb5kdc[29165](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.52.67@KRBTEST.COM for oryx/127.19.52.126@KRBTEST.COM, Server not found in Kerberos database
I20260812 06:55:37.393196 30292 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:37.387327 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.126:44739 (local address 127.19.52.67:42441)
0812 06:55:37.387453 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0812 06:55:37.387470 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0812 06:55:37.387552 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:55:37.387884 (+ 332us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:55:37.387887 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:55:37.388059 (+ 172us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:55:37.388167 (+ 108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:55:37.388173 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:55:37.389111 (+ 938us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:55:37.389117 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:55:37.389685 (+ 568us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:55:37.389694 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:55:37.390160 (+ 466us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:55:37.390163 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:55:37.390267 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:55:37.390807 (+ 540us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 06:55:37.390823 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:55:37.393029 (+ 2206us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.19.52.126:44739: Server oryx/127.19.52.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":55}
W20260812 06:55:37.393299 29990 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 06:55:37.387563 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.52.67:42441 (local address 127.19.52.126:44739)
0812 06:55:37.387687 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0812 06:55:37.387690 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 06:55:37.387701 (+ 11us) server_negotiation.cc:407] Connection header received
0812 06:55:37.387731 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:55:37.387732 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:55:37.387766 (+ 34us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:55:37.387861 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:55:37.388289 (+ 428us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:37.388991 (+ 702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:37.389808 (+ 817us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:37.390035 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:37.390152 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:55:37.393176 (+ 3024us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.19.52.67:42441: BlockingRecv error: recv got EOF from 127.19.52.67:42441 (error 108)
Metrics: {"server-negotiator.queue_time_us":49}
Aug 12 06:55:37 dist-test-slave-65mx krb5kdc[29165](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.52.67@KRBTEST.COM for oryx/127.19.52.126@KRBTEST.COM, Server not found in Kerberos database
Aug 12 06:55:37 dist-test-slave-65mx krb5kdc[29165](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.52.67@KRBTEST.COM for oryx/127.19.52.126@KRBTEST.COM, Server not found in Kerberos database
I20260812 06:55:37.399844 30292 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:37.393845 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.126:44739 (local address 127.19.52.67:58457)
0812 06:55:37.394004 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0812 06:55:37.394020 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0812 06:55:37.394114 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:55:37.394377 (+ 263us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:55:37.394379 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:55:37.394531 (+ 152us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:55:37.394616 (+ 85us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:55:37.394621 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:55:37.395498 (+ 877us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:55:37.395501 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:55:37.396168 (+ 667us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:55:37.396177 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:55:37.396695 (+ 518us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:55:37.396698 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:55:37.396793 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:55:37.397304 (+ 511us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 06:55:37.397316 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:55:37.399690 (+ 2374us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.19.52.126:44739: Server oryx/127.19.52.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":65}
W20260812 06:55:37.399911 29990 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 06:55:37.393974 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.52.67:58457 (local address 127.19.52.126:44739)
0812 06:55:37.394145 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0812 06:55:37.394148 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 06:55:37.394162 (+ 14us) server_negotiation.cc:407] Connection header received
0812 06:55:37.394198 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:55:37.394200 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:55:37.394234 (+ 34us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:55:37.394303 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:55:37.394774 (+ 471us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:37.395369 (+ 595us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:37.396295 (+ 926us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:37.396514 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:37.396714 (+ 200us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:55:37.399801 (+ 3087us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.19.52.67:58457: BlockingRecv error: recv got EOF from 127.19.52.67:58457 (error 108)
Metrics: {"server-negotiator.queue_time_us":104}
W20260812 06:55:37.400214 30286 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
Aug 12 06:55:37 dist-test-slave-65mx krb5kdc[29165](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.52.65@KRBTEST.COM for oryx/127.19.52.126@KRBTEST.COM, Server not found in Kerberos database
Aug 12 06:55:37 dist-test-slave-65mx krb5kdc[29165](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.52.65@KRBTEST.COM for oryx/127.19.52.126@KRBTEST.COM, Server not found in Kerberos database
I20260812 06:55:37.971746 30293 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:37.965104 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.126:44739 (local address 127.19.52.65:41279)
0812 06:55:37.965378 (+ 274us) negotiation.cc:107] Waiting for socket to connect
0812 06:55:37.965391 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 06:55:37.965486 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:55:37.965803 (+ 317us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:55:37.965806 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:55:37.965999 (+ 193us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:55:37.966139 (+ 140us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:55:37.966146 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:55:37.967097 (+ 951us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:55:37.967100 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:55:37.967874 (+ 774us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:55:37.967885 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:55:37.968399 (+ 514us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:55:37.968402 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:55:37.968472 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:55:37.969105 (+ 633us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 06:55:37.969127 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:55:37.971606 (+ 2479us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.19.52.126:44739: Server oryx/127.19.52.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":187,"thread_start_us":81,"threads_started":1}
W20260812 06:55:37.971938 30294 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 06:55:37.965166 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.52.65:41279 (local address 127.19.52.126:44739)
0812 06:55:37.965419 (+ 253us) server_negotiation.cc:206] Beginning negotiation
0812 06:55:37.965422 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 06:55:37.965440 (+ 18us) server_negotiation.cc:407] Connection header received
0812 06:55:37.965623 (+ 183us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:55:37.965626 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:55:37.965676 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:55:37.965764 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:55:37.966272 (+ 508us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:37.966976 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:37.968049 (+ 1073us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:37.968278 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:37.968372 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:55:37.971783 (+ 3411us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.19.52.65:41279: BlockingRecv error: recv got EOF from 127.19.52.65:41279 (error 108)
Metrics: {"server-negotiator.queue_time_us":175,"thread_start_us":78,"threads_started":1}
Aug 12 06:55:38 dist-test-slave-65mx krb5kdc[29165](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.52.66@KRBTEST.COM for oryx/127.19.52.126@KRBTEST.COM, Server not found in Kerberos database
Aug 12 06:55:38 dist-test-slave-65mx krb5kdc[29165](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.52.66@KRBTEST.COM for oryx/127.19.52.126@KRBTEST.COM, Server not found in Kerberos database
I20260812 06:55:38.196156 30296 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:38.189859 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.126:44739 (local address 127.19.52.66:59117)
0812 06:55:38.190177 (+ 318us) negotiation.cc:107] Waiting for socket to connect
0812 06:55:38.190197 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0812 06:55:38.190331 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:55:38.190748 (+ 417us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:55:38.190752 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:55:38.191011 (+ 259us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:55:38.191178 (+ 167us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:55:38.191189 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:55:38.192190 (+ 1001us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:55:38.192196 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:55:38.192860 (+ 664us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:55:38.192869 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:55:38.193356 (+ 487us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:55:38.193360 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:55:38.193447 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:55:38.194026 (+ 579us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 06:55:38.194039 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:55:38.195967 (+ 1928us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.19.52.126:44739: Server oryx/127.19.52.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":227,"thread_start_us":102,"threads_started":1}
W20260812 06:55:38.196220 30294 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 06:55:38.189992 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.52.66:59117 (local address 127.19.52.126:44739)
0812 06:55:38.190194 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0812 06:55:38.190200 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 06:55:38.190303 (+ 103us) server_negotiation.cc:407] Connection header received
0812 06:55:38.190459 (+ 156us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:55:38.190466 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:55:38.190551 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:55:38.190668 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:55:38.191284 (+ 616us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:38.192080 (+ 796us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:38.192982 (+ 902us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:38.193236 (+ 254us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:38.193351 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:55:38.196095 (+ 2744us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.19.52.66:59117: BlockingRecv error: recv got EOF from 127.19.52.66:59117 (error 108)
Metrics: {"server-negotiator.queue_time_us":80}
Aug 12 06:55:38 dist-test-slave-65mx krb5kdc[29165](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.52.67@KRBTEST.COM for oryx/127.19.52.126@KRBTEST.COM, Server not found in Kerberos database
Aug 12 06:55:38 dist-test-slave-65mx krb5kdc[29165](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.52.67@KRBTEST.COM for oryx/127.19.52.126@KRBTEST.COM, Server not found in Kerberos database
I20260812 06:55:38.406804 30297 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:38.400965 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.126:44739 (local address 127.19.52.67:53183)
0812 06:55:38.401323 (+ 358us) negotiation.cc:107] Waiting for socket to connect
0812 06:55:38.401337 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 06:55:38.401481 (+ 144us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:55:38.401836 (+ 355us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:55:38.401838 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:55:38.402032 (+ 194us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:55:38.402137 (+ 105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:55:38.402143 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:55:38.402948 (+ 805us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:55:38.402951 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:55:38.403538 (+ 587us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:55:38.403547 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:55:38.404040 (+ 493us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:55:38.404042 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:55:38.404116 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:55:38.404737 (+ 621us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 06:55:38.404749 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:55:38.406673 (+ 1924us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.19.52.126:44739: Server oryx/127.19.52.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":272,"thread_start_us":127,"threads_started":1}
W20260812 06:55:38.407038 30294 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 06:55:38.401092 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.52.67:53183 (local address 127.19.52.126:44739)
0812 06:55:38.401262 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0812 06:55:38.401267 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 06:55:38.401422 (+ 155us) server_negotiation.cc:407] Connection header received
0812 06:55:38.401622 (+ 200us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:55:38.401627 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:55:38.401684 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:55:38.401777 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:55:38.402273 (+ 496us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:38.402834 (+ 561us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:38.403684 (+ 850us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:38.403919 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:38.404022 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:55:38.406891 (+ 2869us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.19.52.67:53183: BlockingRecv error: recv got EOF from 127.19.52.67:53183 (error 108)
Metrics: {"server-negotiator.queue_time_us":78}
Aug 12 06:55:38 dist-test-slave-65mx krb5kdc[29165](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.52.65@KRBTEST.COM for oryx/127.19.52.126@KRBTEST.COM, Server not found in Kerberos database
Aug 12 06:55:38 dist-test-slave-65mx krb5kdc[29165](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.52.65@KRBTEST.COM for oryx/127.19.52.126@KRBTEST.COM, Server not found in Kerberos database
I20260812 06:55:38.978771 30299 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:38.972845 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.126:44739 (local address 127.19.52.65:49321)
0812 06:55:38.973172 (+ 327us) negotiation.cc:107] Waiting for socket to connect
0812 06:55:38.973195 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0812 06:55:38.973294 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:55:38.973606 (+ 312us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:55:38.973609 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:55:38.973820 (+ 211us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:55:38.973944 (+ 124us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:55:38.973950 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:55:38.974775 (+ 825us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:55:38.974779 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:55:38.975350 (+ 571us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:55:38.975360 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:55:38.975909 (+ 549us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:55:38.975912 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:55:38.975985 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:55:38.976635 (+ 650us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 06:55:38.976649 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:55:38.978617 (+ 1968us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.19.52.126:44739: Server oryx/127.19.52.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":224,"thread_start_us":77,"threads_started":1}
W20260812 06:55:38.979113 30300 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 06:55:38.972912 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.52.65:49321 (local address 127.19.52.126:44739)
0812 06:55:38.973254 (+ 342us) server_negotiation.cc:206] Beginning negotiation
0812 06:55:38.973258 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 06:55:38.973272 (+ 14us) server_negotiation.cc:407] Connection header received
0812 06:55:38.973383 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:55:38.973385 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:55:38.973428 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:55:38.973526 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:55:38.974084 (+ 558us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:38.974654 (+ 570us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:38.975555 (+ 901us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:38.975791 (+ 236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:38.975885 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:55:38.978845 (+ 2960us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.19.52.65:49321: BlockingRecv error: recv got EOF from 127.19.52.65:49321 (error 108)
Metrics: {"server-negotiator.queue_time_us":274,"thread_start_us":109,"threads_started":1}
Aug 12 06:55:39 dist-test-slave-65mx krb5kdc[29165](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.52.66@KRBTEST.COM for oryx/127.19.52.126@KRBTEST.COM, Server not found in Kerberos database
Aug 12 06:55:39 dist-test-slave-65mx krb5kdc[29165](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.52.66@KRBTEST.COM for oryx/127.19.52.126@KRBTEST.COM, Server not found in Kerberos database
I20260812 06:55:39.203800 30301 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:39.197230 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.126:44739 (local address 127.19.52.66:40785)
0812 06:55:39.197649 (+ 419us) negotiation.cc:107] Waiting for socket to connect
0812 06:55:39.197662 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 06:55:39.197818 (+ 156us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:55:39.198110 (+ 292us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:55:39.198113 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:55:39.198331 (+ 218us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:55:39.198429 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:55:39.198434 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:55:39.199253 (+ 819us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:55:39.199258 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:55:39.199828 (+ 570us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:55:39.199836 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:55:39.200322 (+ 486us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:55:39.200326 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:55:39.200520 (+ 194us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:55:39.201207 (+ 687us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 06:55:39.201220 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:55:39.203258 (+ 2038us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.19.52.126:44739: Server oryx/127.19.52.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":282,"thread_start_us":92,"threads_started":1}
W20260812 06:55:39.203858 30300 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 06:55:39.197369 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.52.66:40785 (local address 127.19.52.126:44739)
0812 06:55:39.197593 (+ 224us) server_negotiation.cc:206] Beginning negotiation
0812 06:55:39.197597 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 06:55:39.197749 (+ 152us) server_negotiation.cc:407] Connection header received
0812 06:55:39.197931 (+ 182us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:55:39.197934 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:55:39.197990 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:55:39.198091 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:55:39.198558 (+ 467us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:39.199131 (+ 573us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:39.199955 (+ 824us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:39.200190 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:39.200300 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:55:39.203736 (+ 3436us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.19.52.66:40785: BlockingRecv error: recv got EOF from 127.19.52.66:40785 (error 108)
Metrics: {"server-negotiator.queue_time_us":110}
Aug 12 06:55:39 dist-test-slave-65mx krb5kdc[29165](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.52.67@KRBTEST.COM for oryx/127.19.52.126@KRBTEST.COM, Server not found in Kerberos database
Aug 12 06:55:39 dist-test-slave-65mx krb5kdc[29165](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.52.67@KRBTEST.COM for oryx/127.19.52.126@KRBTEST.COM, Server not found in Kerberos database
I20260812 06:55:39.415680 30302 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:39.407911 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.126:44739 (local address 127.19.52.67:43253)
0812 06:55:39.408358 (+ 447us) negotiation.cc:107] Waiting for socket to connect
0812 06:55:39.408375 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0812 06:55:39.408516 (+ 141us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:55:39.408929 (+ 413us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:55:39.408934 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:55:39.409209 (+ 275us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:55:39.409362 (+ 153us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:55:39.409372 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:55:39.410762 (+ 1390us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:55:39.410768 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:55:39.411623 (+ 855us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:55:39.411636 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:55:39.412228 (+ 592us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:55:39.412232 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:55:39.412315 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:55:39.413179 (+ 864us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 06:55:39.413195 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:55:39.415485 (+ 2290us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.19.52.126:44739: Server oryx/127.19.52.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":354,"thread_start_us":106,"threads_started":1}
W20260812 06:55:39.416076 30300 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 06:55:39.408129 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.52.67:43253 (local address 127.19.52.126:44739)
0812 06:55:39.408312 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0812 06:55:39.408317 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 06:55:39.408595 (+ 278us) server_negotiation.cc:407] Connection header received
0812 06:55:39.408669 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:55:39.408675 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:55:39.408748 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:55:39.408899 (+ 151us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:55:39.409533 (+ 634us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:39.410609 (+ 1076us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:39.411764 (+ 1155us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:39.412089 (+ 325us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:39.412210 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:55:39.415942 (+ 3732us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.19.52.67:43253: BlockingRecv error: recv got EOF from 127.19.52.67:43253 (error 108)
Metrics: {"server-negotiator.queue_time_us":78}
Aug 12 06:55:39 dist-test-slave-65mx krb5kdc[29165](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.52.65@KRBTEST.COM for oryx/127.19.52.126@KRBTEST.COM, Server not found in Kerberos database
Aug 12 06:55:39 dist-test-slave-65mx krb5kdc[29165](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.52.65@KRBTEST.COM for oryx/127.19.52.126@KRBTEST.COM, Server not found in Kerberos database
I20260812 06:55:39.986804 30303 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:39.979840 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.126:44739 (local address 127.19.52.65:42089)
0812 06:55:39.980227 (+ 387us) negotiation.cc:107] Waiting for socket to connect
0812 06:55:39.980248 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0812 06:55:39.980362 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:55:39.980906 (+ 544us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:55:39.980909 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:55:39.981149 (+ 240us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:55:39.981259 (+ 110us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:55:39.981266 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:55:39.982529 (+ 1263us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:55:39.982536 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:55:39.983353 (+ 817us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:55:39.983366 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:55:39.983969 (+ 603us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:55:39.983973 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:55:39.984048 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:55:39.984673 (+ 625us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 06:55:39.984687 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:55:39.986673 (+ 1986us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.19.52.126:44739: Server oryx/127.19.52.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":280,"thread_start_us":110,"threads_started":1}
W20260812 06:55:39.987051 30304 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 06:55:39.979956 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.52.65:42089 (local address 127.19.52.126:44739)
0812 06:55:39.980659 (+ 703us) server_negotiation.cc:206] Beginning negotiation
0812 06:55:39.980664 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 06:55:39.980680 (+ 16us) server_negotiation.cc:407] Connection header received
0812 06:55:39.980722 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:55:39.980725 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:55:39.980769 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:55:39.980855 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:55:39.981412 (+ 557us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:39.982368 (+ 956us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:39.983559 (+ 1191us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:39.983793 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:39.983889 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:55:39.986907 (+ 3018us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.19.52.65:42089: BlockingRecv error: recv got EOF from 127.19.52.65:42089 (error 108)
Metrics: {"server-negotiator.queue_time_us":409,"thread_start_us":179,"threads_started":1}
Aug 12 06:55:40 dist-test-slave-65mx krb5kdc[29165](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.52.66@KRBTEST.COM for oryx/127.19.52.126@KRBTEST.COM, Server not found in Kerberos database
Aug 12 06:55:40 dist-test-slave-65mx krb5kdc[29165](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.52.66@KRBTEST.COM for oryx/127.19.52.126@KRBTEST.COM, Server not found in Kerberos database
I20260812 06:55:40.211089 30305 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:40.204881 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.126:44739 (local address 127.19.52.66:37057)
0812 06:55:40.205268 (+ 387us) negotiation.cc:107] Waiting for socket to connect
0812 06:55:40.205284 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0812 06:55:40.205401 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:55:40.205713 (+ 312us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:55:40.205716 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:55:40.205909 (+ 193us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:55:40.206022 (+ 113us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:55:40.206028 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:55:40.207043 (+ 1015us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:55:40.207046 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:55:40.207638 (+ 592us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:55:40.207647 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:55:40.208146 (+ 499us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:55:40.208149 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:55:40.208225 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:55:40.208864 (+ 639us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 06:55:40.208877 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:55:40.210963 (+ 2086us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.19.52.126:44739: Server oryx/127.19.52.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":309,"thread_start_us":173,"threads_started":1}
W20260812 06:55:40.211284 30304 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 06:55:40.204988 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.52.66:37057 (local address 127.19.52.126:44739)
0812 06:55:40.205167 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0812 06:55:40.205172 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 06:55:40.205425 (+ 253us) server_negotiation.cc:407] Connection header received
0812 06:55:40.205501 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:55:40.205503 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:55:40.205566 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:55:40.205644 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:55:40.206150 (+ 506us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:40.206909 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:40.207762 (+ 853us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:40.207982 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:40.208069 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:55:40.211141 (+ 3072us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.19.52.66:37057: BlockingRecv error: recv got EOF from 127.19.52.66:37057 (error 108)
Metrics: {"server-negotiator.queue_time_us":67}
Aug 12 06:55:40 dist-test-slave-65mx krb5kdc[29165](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.52.67@KRBTEST.COM for oryx/127.19.52.126@KRBTEST.COM, Server not found in Kerberos database
Aug 12 06:55:40 dist-test-slave-65mx krb5kdc[29165](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.52.67@KRBTEST.COM for oryx/127.19.52.126@KRBTEST.COM, Server not found in Kerberos database
I20260812 06:55:40.427138 30307 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:40.416705 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.126:44739 (local address 127.19.52.67:40305)
0812 06:55:40.417068 (+ 363us) negotiation.cc:107] Waiting for socket to connect
0812 06:55:40.417082 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 06:55:40.417203 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:55:40.417490 (+ 287us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:55:40.417492 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:55:40.417681 (+ 189us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:55:40.417779 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:55:40.417784 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:55:40.418582 (+ 798us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:55:40.418585 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:55:40.419149 (+ 564us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:55:40.419158 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:55:40.419669 (+ 511us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:55:40.419672 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:55:40.419762 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:55:40.424606 (+ 4844us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 06:55:40.424636 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:55:40.426960 (+ 2324us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.19.52.126:44739: Server oryx/127.19.52.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":286,"thread_start_us":122,"threads_started":1}
W20260812 06:55:40.427269 30304 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 06:55:40.416796 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.52.67:40305 (local address 127.19.52.126:44739)
0812 06:55:40.416966 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0812 06:55:40.416971 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 06:55:40.417164 (+ 193us) server_negotiation.cc:407] Connection header received
0812 06:55:40.417277 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:55:40.417279 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:55:40.417327 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:55:40.417400 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:55:40.417909 (+ 509us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:40.418471 (+ 562us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:40.419294 (+ 823us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:40.419533 (+ 239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:40.419662 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:55:40.427156 (+ 7494us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.19.52.67:40305: BlockingRecv error: recv got EOF from 127.19.52.67:40305 (error 108)
Metrics: {"server-negotiator.queue_time_us":63}
Aug 12 06:55:40 dist-test-slave-65mx krb5kdc[29165](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.52.65@KRBTEST.COM for oryx/127.19.52.126@KRBTEST.COM, Server not found in Kerberos database
Aug 12 06:55:40 dist-test-slave-65mx krb5kdc[29165](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.52.65@KRBTEST.COM for oryx/127.19.52.126@KRBTEST.COM, Server not found in Kerberos database
I20260812 06:55:40.993935 30308 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:40.987752 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.126:44739 (local address 127.19.52.65:36651)
0812 06:55:40.988063 (+ 311us) negotiation.cc:107] Waiting for socket to connect
0812 06:55:40.988077 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 06:55:40.988178 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:55:40.988443 (+ 265us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:55:40.988445 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:55:40.988689 (+ 244us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:55:40.988795 (+ 106us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:55:40.988801 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:55:40.989660 (+ 859us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:55:40.989664 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:55:40.990229 (+ 565us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:55:40.990238 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:55:40.990717 (+ 479us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:55:40.990720 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:55:40.990784 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:55:40.991341 (+ 557us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 06:55:40.991353 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:55:40.993765 (+ 2412us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.19.52.126:44739: Server oryx/127.19.52.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":214,"thread_start_us":118,"threads_started":1}
W20260812 06:55:40.994120 30309 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 06:55:40.987932 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.52.65:36651 (local address 127.19.52.126:44739)
0812 06:55:40.988191 (+ 259us) server_negotiation.cc:206] Beginning negotiation
0812 06:55:40.988195 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 06:55:40.988211 (+ 16us) server_negotiation.cc:407] Connection header received
0812 06:55:40.988257 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:55:40.988259 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:55:40.988306 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:55:40.988427 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:55:40.988952 (+ 525us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:40.989537 (+ 585us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:40.990350 (+ 813us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:40.990598 (+ 248us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:40.990684 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:55:40.993957 (+ 3273us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.19.52.65:36651: BlockingRecv error: recv got EOF from 127.19.52.65:36651 (error 108)
Metrics: {"server-negotiator.queue_time_us":191,"thread_start_us":92,"threads_started":1}
Aug 12 06:55:41 dist-test-slave-65mx krb5kdc[29165](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.52.66@KRBTEST.COM for oryx/127.19.52.126@KRBTEST.COM, Server not found in Kerberos database
Aug 12 06:55:41 dist-test-slave-65mx krb5kdc[29165](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.52.66@KRBTEST.COM for oryx/127.19.52.126@KRBTEST.COM, Server not found in Kerberos database
I20260812 06:55:41.217756 30310 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:41.212037 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.126:44739 (local address 127.19.52.66:45875)
0812 06:55:41.212338 (+ 301us) negotiation.cc:107] Waiting for socket to connect
0812 06:55:41.212357 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0812 06:55:41.212511 (+ 154us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:55:41.212856 (+ 345us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:55:41.212859 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:55:41.213099 (+ 240us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:55:41.213198 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:55:41.213203 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:55:41.213968 (+ 765us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:55:41.213973 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:55:41.214526 (+ 553us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:55:41.214533 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:55:41.214984 (+ 451us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:55:41.214987 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:55:41.215081 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:55:41.215660 (+ 579us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 06:55:41.215672 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:55:41.217629 (+ 1957us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.19.52.126:44739: Server oryx/127.19.52.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":219,"thread_start_us":113,"threads_started":1}
W20260812 06:55:41.217885 30309 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 06:55:41.212141 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.52.66:45875 (local address 127.19.52.126:44739)
0812 06:55:41.212313 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0812 06:55:41.212318 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 06:55:41.212448 (+ 130us) server_negotiation.cc:407] Connection header received
0812 06:55:41.212683 (+ 235us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:55:41.212685 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:55:41.212731 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:55:41.212824 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:55:41.213324 (+ 500us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:41.213853 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:41.214649 (+ 796us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:41.214868 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:41.214979 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:55:41.217770 (+ 2791us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.19.52.66:45875: BlockingRecv error: recv got EOF from 127.19.52.66:45875 (error 108)
Metrics: {"server-negotiator.queue_time_us":83}
Aug 12 06:55:41 dist-test-slave-65mx krb5kdc[29165](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.52.67@KRBTEST.COM for oryx/127.19.52.126@KRBTEST.COM, Server not found in Kerberos database
Aug 12 06:55:41 dist-test-slave-65mx krb5kdc[29165](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.52.67@KRBTEST.COM for oryx/127.19.52.126@KRBTEST.COM, Server not found in Kerberos database
I20260812 06:55:41.434223 30311 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:41.428164 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.126:44739 (local address 127.19.52.67:53441)
0812 06:55:41.428541 (+ 377us) negotiation.cc:107] Waiting for socket to connect
0812 06:55:41.428560 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0812 06:55:41.428753 (+ 193us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:55:41.429049 (+ 296us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:55:41.429052 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:55:41.429277 (+ 225us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:55:41.429370 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:55:41.429376 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:55:41.430151 (+ 775us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:55:41.430156 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:55:41.430717 (+ 561us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:55:41.430725 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:55:41.431215 (+ 490us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:55:41.431219 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:55:41.431306 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:55:41.431901 (+ 595us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 06:55:41.431922 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:55:41.434011 (+ 2089us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.19.52.126:44739: Server oryx/127.19.52.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":292,"thread_start_us":129,"threads_started":1}
W20260812 06:55:41.434266 30309 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 06:55:41.428353 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.52.67:53441 (local address 127.19.52.126:44739)
0812 06:55:41.428560 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0812 06:55:41.428566 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 06:55:41.428683 (+ 117us) server_negotiation.cc:407] Connection header received
0812 06:55:41.428864 (+ 181us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:55:41.428867 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:55:41.428918 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:55:41.429030 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:55:41.429498 (+ 468us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:41.430036 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:41.430844 (+ 808us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:41.431086 (+ 242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:41.431198 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:55:41.434161 (+ 2963us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.19.52.67:53441: BlockingRecv error: recv got EOF from 127.19.52.67:53441 (error 108)
Metrics: {"server-negotiator.queue_time_us":83}
Aug 12 06:55:41 dist-test-slave-65mx krb5kdc[29165](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.52.65@KRBTEST.COM for oryx/127.19.52.126@KRBTEST.COM, Server not found in Kerberos database
Aug 12 06:55:42 dist-test-slave-65mx krb5kdc[29165](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.52.65@KRBTEST.COM for oryx/127.19.52.126@KRBTEST.COM, Server not found in Kerberos database
I20260812 06:55:42.001647 30313 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:41.995144 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.126:44739 (local address 127.19.52.65:47629)
0812 06:55:41.995484 (+ 340us) negotiation.cc:107] Waiting for socket to connect
0812 06:55:41.995498 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 06:55:41.995622 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:55:41.996001 (+ 379us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:55:41.996005 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:55:41.996266 (+ 261us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:55:41.996405 (+ 139us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:55:41.996415 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:55:41.997427 (+ 1012us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:55:41.997430 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:55:41.998089 (+ 659us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:55:41.998099 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:55:41.998557 (+ 458us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:55:41.998560 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:55:41.998633 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:55:41.999254 (+ 621us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 06:55:41.999268 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:55:42.001456 (+ 2188us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.19.52.126:44739: Server oryx/127.19.52.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":220,"thread_start_us":120,"threads_started":1}
W20260812 06:55:42.001778 30314 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 06:55:41.995302 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.52.65:47629 (local address 127.19.52.126:44739)
0812 06:55:41.995633 (+ 331us) server_negotiation.cc:206] Beginning negotiation
0812 06:55:41.995639 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 06:55:41.995657 (+ 18us) server_negotiation.cc:407] Connection header received
0812 06:55:41.995723 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:55:41.995726 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:55:41.995769 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:55:41.995892 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:55:41.996614 (+ 722us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:41.997317 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:41.998205 (+ 888us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:41.998425 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:41.998506 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:55:42.001623 (+ 3117us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.19.52.65:47629: BlockingRecv error: recv got EOF from 127.19.52.65:47629 (error 108)
Metrics: {"server-negotiator.queue_time_us":243,"thread_start_us":129,"threads_started":1}
Aug 12 06:55:42 dist-test-slave-65mx krb5kdc[29165](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.52.66@KRBTEST.COM for oryx/127.19.52.126@KRBTEST.COM, Server not found in Kerberos database
Aug 12 06:55:42 dist-test-slave-65mx krb5kdc[29165](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.52.66@KRBTEST.COM for oryx/127.19.52.126@KRBTEST.COM, Server not found in Kerberos database
I20260812 06:55:42.225129 30315 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:42.218708 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.126:44739 (local address 127.19.52.66:55573)
0812 06:55:42.219139 (+ 431us) negotiation.cc:107] Waiting for socket to connect
0812 06:55:42.219152 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 06:55:42.219241 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:55:42.219574 (+ 333us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:55:42.219577 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:55:42.219797 (+ 220us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:55:42.219923 (+ 126us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:55:42.219929 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:55:42.220816 (+ 887us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:55:42.220819 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:55:42.221461 (+ 642us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:55:42.221470 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:55:42.221962 (+ 492us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:55:42.221965 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:55:42.222036 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:55:42.222682 (+ 646us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 06:55:42.222695 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:55:42.224998 (+ 2303us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.19.52.126:44739: Server oryx/127.19.52.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":344,"thread_start_us":146,"threads_started":1}
W20260812 06:55:42.225380 30314 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 06:55:42.218890 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.52.66:55573 (local address 127.19.52.126:44739)
0812 06:55:42.219177 (+ 287us) server_negotiation.cc:206] Beginning negotiation
0812 06:55:42.219181 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 06:55:42.219199 (+ 18us) server_negotiation.cc:407] Connection header received
0812 06:55:42.219335 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:55:42.219337 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:55:42.219384 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:55:42.219474 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:55:42.220068 (+ 594us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:42.220696 (+ 628us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:42.221601 (+ 905us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:42.221836 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:42.221984 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:55:42.225256 (+ 3272us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.19.52.66:55573: BlockingRecv error: recv got EOF from 127.19.52.66:55573 (error 108)
Metrics: {"server-negotiator.queue_time_us":130}
Aug 12 06:55:42 dist-test-slave-65mx krb5kdc[29165](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.52.67@KRBTEST.COM for oryx/127.19.52.126@KRBTEST.COM, Server not found in Kerberos database
Aug 12 06:55:42 dist-test-slave-65mx krb5kdc[29165](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.52.67@KRBTEST.COM for oryx/127.19.52.126@KRBTEST.COM, Server not found in Kerberos database
I20260812 06:55:42.442070 30316 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:42.435234 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.126:44739 (local address 127.19.52.67:42591)
0812 06:55:42.435736 (+ 502us) negotiation.cc:107] Waiting for socket to connect
0812 06:55:42.435752 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0812 06:55:42.435913 (+ 161us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:55:42.436216 (+ 303us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:55:42.436219 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:55:42.436649 (+ 430us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:55:42.436764 (+ 115us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:55:42.436772 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:55:42.437582 (+ 810us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:55:42.437586 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:55:42.438150 (+ 564us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:55:42.438158 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:55:42.438635 (+ 477us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:55:42.438638 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:55:42.438726 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:55:42.439305 (+ 579us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 06:55:42.439316 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:55:42.441878 (+ 2562us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.19.52.126:44739: Server oryx/127.19.52.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":419,"thread_start_us":254,"threads_started":1}
W20260812 06:55:42.442131 30314 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 06:55:42.435509 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.52.67:42591 (local address 127.19.52.126:44739)
0812 06:55:42.435703 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0812 06:55:42.435708 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 06:55:42.435851 (+ 143us) server_negotiation.cc:407] Connection header received
0812 06:55:42.436021 (+ 170us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:55:42.436025 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:55:42.436087 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:55:42.436184 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:55:42.436897 (+ 713us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:42.437450 (+ 553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:42.438286 (+ 836us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:42.438513 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:42.438627 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:55:42.442018 (+ 3391us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.19.52.67:42591: BlockingRecv error: recv got EOF from 127.19.52.67:42591 (error 108)
Metrics: {"server-negotiator.queue_time_us":86}
Aug 12 06:55:43 dist-test-slave-65mx krb5kdc[29165](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.52.65@KRBTEST.COM for oryx/127.19.52.126@KRBTEST.COM, Server not found in Kerberos database
Aug 12 06:55:43 dist-test-slave-65mx krb5kdc[29165](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.52.65@KRBTEST.COM for oryx/127.19.52.126@KRBTEST.COM, Server not found in Kerberos database
I20260812 06:55:43.009088 30317 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:43.002885 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.126:44739 (local address 127.19.52.65:37165)
0812 06:55:43.003239 (+ 354us) negotiation.cc:107] Waiting for socket to connect
0812 06:55:43.003257 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0812 06:55:43.003464 (+ 207us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:55:43.003830 (+ 366us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:55:43.003834 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:55:43.004079 (+ 245us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:55:43.004183 (+ 104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:55:43.004189 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:55:43.005215 (+ 1026us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:55:43.005218 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:55:43.005781 (+ 563us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:55:43.005789 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:55:43.006310 (+ 521us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:55:43.006313 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:55:43.006378 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:55:43.006929 (+ 551us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 06:55:43.006940 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:55:43.008950 (+ 2010us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.19.52.126:44739: Server oryx/127.19.52.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":257,"thread_start_us":113,"threads_started":1}
W20260812 06:55:43.009347 30318 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 06:55:43.002913 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.52.65:37165 (local address 127.19.52.126:44739)
0812 06:55:43.003240 (+ 327us) server_negotiation.cc:206] Beginning negotiation
0812 06:55:43.003246 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 06:55:43.003390 (+ 144us) server_negotiation.cc:407] Connection header received
0812 06:55:43.003602 (+ 212us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:55:43.003605 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:55:43.003667 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:55:43.003780 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:55:43.004328 (+ 548us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:43.005107 (+ 779us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:43.005900 (+ 793us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:43.006199 (+ 299us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:43.006292 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:55:43.009183 (+ 2891us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.19.52.65:37165: BlockingRecv error: recv got EOF from 127.19.52.65:37165 (error 108)
Metrics: {"server-negotiator.queue_time_us":263,"thread_start_us":132,"threads_started":1}
Aug 12 06:55:43 dist-test-slave-65mx krb5kdc[29165](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.52.66@KRBTEST.COM for oryx/127.19.52.126@KRBTEST.COM, Server not found in Kerberos database
Aug 12 06:55:43 dist-test-slave-65mx krb5kdc[29165](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.52.66@KRBTEST.COM for oryx/127.19.52.126@KRBTEST.COM, Server not found in Kerberos database
I20260812 06:55:43.232324 30319 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:43.226143 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.126:44739 (local address 127.19.52.66:41099)
0812 06:55:43.226530 (+ 387us) negotiation.cc:107] Waiting for socket to connect
0812 06:55:43.226551 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0812 06:55:43.226697 (+ 146us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:55:43.226986 (+ 289us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:55:43.226990 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:55:43.227259 (+ 269us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:55:43.227356 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:55:43.227362 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:55:43.228176 (+ 814us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:55:43.228182 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:55:43.228878 (+ 696us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:55:43.228890 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:55:43.229446 (+ 556us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:55:43.229455 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:55:43.229551 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:55:43.230153 (+ 602us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 06:55:43.230165 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:55:43.232141 (+ 1976us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.19.52.126:44739: Server oryx/127.19.52.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":320,"thread_start_us":118,"threads_started":1}
W20260812 06:55:43.232419 30318 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 06:55:43.226203 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.52.66:41099 (local address 127.19.52.126:44739)
0812 06:55:43.226369 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0812 06:55:43.226374 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 06:55:43.226640 (+ 266us) server_negotiation.cc:407] Connection header received
0812 06:55:43.226810 (+ 170us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:55:43.226813 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:55:43.226870 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:55:43.226957 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:55:43.227502 (+ 545us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:43.228052 (+ 550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:43.229033 (+ 981us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:43.229297 (+ 264us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:43.229415 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:55:43.232295 (+ 2880us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.19.52.66:41099: BlockingRecv error: recv got EOF from 127.19.52.66:41099 (error 108)
Metrics: {"server-negotiator.queue_time_us":68}
Aug 12 06:55:43 dist-test-slave-65mx krb5kdc[29165](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.52.67@KRBTEST.COM for oryx/127.19.52.126@KRBTEST.COM, Server not found in Kerberos database
Aug 12 06:55:43 dist-test-slave-65mx krb5kdc[29165](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.52.67@KRBTEST.COM for oryx/127.19.52.126@KRBTEST.COM, Server not found in Kerberos database
I20260812 06:55:43.449662 30320 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:43.443115 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.126:44739 (local address 127.19.52.67:50137)
0812 06:55:43.443402 (+ 287us) negotiation.cc:107] Waiting for socket to connect
0812 06:55:43.443416 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 06:55:43.443532 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:55:43.443972 (+ 440us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:55:43.443975 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:55:43.444175 (+ 200us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:55:43.444291 (+ 116us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:55:43.444297 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:55:43.445460 (+ 1163us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:55:43.445463 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:55:43.446054 (+ 591us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:55:43.446064 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:55:43.446601 (+ 537us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:55:43.446604 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:55:43.446699 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:55:43.447350 (+ 651us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 06:55:43.447363 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:55:43.449541 (+ 2178us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.19.52.126:44739: Server oryx/127.19.52.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":197,"thread_start_us":103,"threads_started":1}
W20260812 06:55:43.449859 30318 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 06:55:43.443201 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.52.67:50137 (local address 127.19.52.126:44739)
0812 06:55:43.443382 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0812 06:55:43.443386 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 06:55:43.443584 (+ 198us) server_negotiation.cc:407] Connection header received
0812 06:55:43.443693 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:55:43.443696 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:55:43.443785 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:55:43.443926 (+ 141us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:55:43.444501 (+ 575us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:43.445271 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:43.446230 (+ 959us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:43.446470 (+ 240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:43.446592 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:55:43.449719 (+ 3127us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.19.52.67:50137: BlockingRecv error: recv got EOF from 127.19.52.67:50137 (error 108)
Metrics: {"server-negotiator.queue_time_us":86}
Aug 12 06:55:44 dist-test-slave-65mx krb5kdc[29165](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.52.65@KRBTEST.COM for oryx/127.19.52.126@KRBTEST.COM, Server not found in Kerberos database
Aug 12 06:55:44 dist-test-slave-65mx krb5kdc[29165](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.52.65@KRBTEST.COM for oryx/127.19.52.126@KRBTEST.COM, Server not found in Kerberos database
I20260812 06:55:44.016662 30321 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:44.010005 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.126:44739 (local address 127.19.52.65:52949)
0812 06:55:44.010464 (+ 459us) negotiation.cc:107] Waiting for socket to connect
0812 06:55:44.010478 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 06:55:44.010582 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:55:44.010840 (+ 258us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:55:44.010842 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:55:44.011033 (+ 191us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:55:44.011209 (+ 176us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:55:44.011216 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:55:44.012185 (+ 969us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:55:44.012189 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:55:44.012815 (+ 626us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:55:44.012825 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:55:44.013289 (+ 464us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:55:44.013292 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:55:44.013356 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:55:44.013977 (+ 621us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 06:55:44.013990 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:55:44.016448 (+ 2458us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.19.52.126:44739: Server oryx/127.19.52.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":364,"thread_start_us":140,"threads_started":1}
W20260812 06:55:44.016841 30322 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 06:55:44.010188 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.52.65:52949 (local address 127.19.52.126:44739)
0812 06:55:44.010538 (+ 350us) server_negotiation.cc:206] Beginning negotiation
0812 06:55:44.010549 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0812 06:55:44.010567 (+ 18us) server_negotiation.cc:407] Connection header received
0812 06:55:44.010670 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:55:44.010672 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:55:44.010717 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:55:44.010799 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:55:44.011344 (+ 545us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:44.012033 (+ 689us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:44.012938 (+ 905us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:44.013177 (+ 239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:44.013263 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:55:44.016678 (+ 3415us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.19.52.65:52949: BlockingRecv error: recv got EOF from 127.19.52.65:52949 (error 108)
Metrics: {"server-negotiator.queue_time_us":232,"thread_start_us":136,"threads_started":1}
Aug 12 06:55:44 dist-test-slave-65mx krb5kdc[29165](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.52.66@KRBTEST.COM for oryx/127.19.52.126@KRBTEST.COM, Server not found in Kerberos database
Aug 12 06:55:44 dist-test-slave-65mx krb5kdc[29165](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.52.66@KRBTEST.COM for oryx/127.19.52.126@KRBTEST.COM, Server not found in Kerberos database
I20260812 06:55:44.240845 30323 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:44.233612 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.126:44739 (local address 127.19.52.66:47541)
0812 06:55:44.234046 (+ 434us) negotiation.cc:107] Waiting for socket to connect
0812 06:55:44.234067 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0812 06:55:44.234244 (+ 177us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:55:44.234567 (+ 323us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:55:44.234571 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:55:44.234818 (+ 247us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:55:44.234922 (+ 104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:55:44.234928 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:55:44.235785 (+ 857us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:55:44.235791 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:55:44.236423 (+ 632us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:55:44.236432 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:55:44.237023 (+ 591us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:55:44.237027 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:55:44.237127 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:55:44.237880 (+ 753us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 06:55:44.237892 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:55:44.240673 (+ 2781us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.19.52.126:44739: Server oryx/127.19.52.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":336,"thread_start_us":93,"threads_started":1}
W20260812 06:55:44.241078 30322 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 06:55:44.233725 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.52.66:47541 (local address 127.19.52.126:44739)
0812 06:55:44.234000 (+ 275us) server_negotiation.cc:206] Beginning negotiation
0812 06:55:44.234006 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 06:55:44.234171 (+ 165us) server_negotiation.cc:407] Connection header received
0812 06:55:44.234367 (+ 196us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:55:44.234370 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:55:44.234431 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:55:44.234538 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:55:44.235065 (+ 527us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:44.235655 (+ 590us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:44.236592 (+ 937us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:44.236848 (+ 256us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:44.237002 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:55:44.240909 (+ 3907us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.19.52.66:47541: BlockingRecv error: recv got EOF from 127.19.52.66:47541 (error 108)
Metrics: {"server-negotiator.queue_time_us":109}
Aug 12 06:55:44 dist-test-slave-65mx krb5kdc[29165](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.52.67@KRBTEST.COM for oryx/127.19.52.126@KRBTEST.COM, Server not found in Kerberos database
Aug 12 06:55:44 dist-test-slave-65mx krb5kdc[29165](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.52.67@KRBTEST.COM for oryx/127.19.52.126@KRBTEST.COM, Server not found in Kerberos database
I20260812 06:55:44.457705 30324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:44.450642 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.126:44739 (local address 127.19.52.67:42569)
0812 06:55:44.451008 (+ 366us) negotiation.cc:107] Waiting for socket to connect
0812 06:55:44.451023 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 06:55:44.451126 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:55:44.451552 (+ 426us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:55:44.451555 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:55:44.451765 (+ 210us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:55:44.451880 (+ 115us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:55:44.451886 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:55:44.452874 (+ 988us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:55:44.452877 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:55:44.453517 (+ 640us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:55:44.453529 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:55:44.454083 (+ 554us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:55:44.454087 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:55:44.454180 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:55:44.455087 (+ 907us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 06:55:44.455123 (+ 36us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:55:44.457568 (+ 2445us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.19.52.126:44739: Server oryx/127.19.52.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":270,"thread_start_us":124,"threads_started":1}
W20260812 06:55:44.457955 30322 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 06:55:44.450819 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.52.67:42569 (local address 127.19.52.126:44739)
0812 06:55:44.451149 (+ 330us) server_negotiation.cc:206] Beginning negotiation
0812 06:55:44.451157 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 06:55:44.451178 (+ 21us) server_negotiation.cc:407] Connection header received
0812 06:55:44.451232 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:55:44.451235 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:55:44.451283 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:55:44.451376 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:55:44.452055 (+ 679us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:44.452736 (+ 681us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:44.453673 (+ 937us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:44.453962 (+ 289us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:44.454057 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:55:44.457770 (+ 3713us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.19.52.67:42569: BlockingRecv error: recv got EOF from 127.19.52.67:42569 (error 108)
Metrics: {"server-negotiator.queue_time_us":213}
Aug 12 06:55:45 dist-test-slave-65mx krb5kdc[29165](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.52.65@KRBTEST.COM for oryx/127.19.52.126@KRBTEST.COM, Server not found in Kerberos database
Aug 12 06:55:45 dist-test-slave-65mx krb5kdc[29165](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.52.65@KRBTEST.COM for oryx/127.19.52.126@KRBTEST.COM, Server not found in Kerberos database
I20260812 06:55:45.023859 30325 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:45.017728 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.126:44739 (local address 127.19.52.65:39767)
0812 06:55:45.018090 (+ 362us) negotiation.cc:107] Waiting for socket to connect
0812 06:55:45.018105 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 06:55:45.018255 (+ 150us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:55:45.018616 (+ 361us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:55:45.018618 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:55:45.018882 (+ 264us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:55:45.019047 (+ 165us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:55:45.019058 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:55:45.019833 (+ 775us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:55:45.019835 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:55:45.020438 (+ 603us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:55:45.020447 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:55:45.020950 (+ 503us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:55:45.020953 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:55:45.021023 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:55:45.021577 (+ 554us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 06:55:45.021591 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:55:45.023726 (+ 2135us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.19.52.126:44739: Server oryx/127.19.52.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":232,"thread_start_us":95,"threads_started":1}
W20260812 06:55:45.024096 30326 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 06:55:45.017908 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.52.65:39767 (local address 127.19.52.126:44739)
0812 06:55:45.018177 (+ 269us) server_negotiation.cc:206] Beginning negotiation
0812 06:55:45.018186 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 06:55:45.018200 (+ 14us) server_negotiation.cc:407] Connection header received
0812 06:55:45.018387 (+ 187us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:55:45.018389 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:55:45.018451 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:55:45.018565 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:55:45.019177 (+ 612us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:45.019730 (+ 553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:45.020585 (+ 855us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:45.020812 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:45.020907 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:55:45.023921 (+ 3014us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.19.52.65:39767: BlockingRecv error: recv got EOF from 127.19.52.65:39767 (error 108)
Metrics: {"server-negotiator.queue_time_us":181,"thread_start_us":68,"threads_started":1}
Aug 12 06:55:45 dist-test-slave-65mx krb5kdc[29165](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.52.66@KRBTEST.COM for oryx/127.19.52.126@KRBTEST.COM, Server not found in Kerberos database
Aug 12 06:55:45 dist-test-slave-65mx krb5kdc[29165](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.52.66@KRBTEST.COM for oryx/127.19.52.126@KRBTEST.COM, Server not found in Kerberos database
I20260812 06:55:45.247586 30327 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:45.241664 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.126:44739 (local address 127.19.52.66:56093)
0812 06:55:45.241971 (+ 307us) negotiation.cc:107] Waiting for socket to connect
0812 06:55:45.241984 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 06:55:45.242089 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:55:45.242418 (+ 329us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:55:45.242420 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:55:45.242636 (+ 216us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:55:45.242733 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:55:45.242738 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:55:45.243717 (+ 979us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:55:45.243721 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:55:45.244326 (+ 605us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:55:45.244336 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:55:45.244881 (+ 545us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:55:45.244885 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:55:45.244970 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:55:45.245524 (+ 554us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 06:55:45.245536 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:55:45.247458 (+ 1922us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.19.52.126:44739: Server oryx/127.19.52.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":214,"thread_start_us":119,"threads_started":1}
W20260812 06:55:45.247884 30326 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 06:55:45.241811 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.52.66:56093 (local address 127.19.52.126:44739)
0812 06:55:45.241973 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0812 06:55:45.241977 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 06:55:45.242040 (+ 63us) server_negotiation.cc:407] Connection header received
0812 06:55:45.242223 (+ 183us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:55:45.242226 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:55:45.242277 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:55:45.242354 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:55:45.242857 (+ 503us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:45.243589 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:45.244465 (+ 876us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:45.244748 (+ 283us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:45.244848 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:55:45.247762 (+ 2914us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.19.52.66:56093: BlockingRecv error: recv got EOF from 127.19.52.66:56093 (error 108)
Metrics: {"server-negotiator.queue_time_us":73}
Aug 12 06:55:45 dist-test-slave-65mx krb5kdc[29165](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.52.67@KRBTEST.COM for oryx/127.19.52.126@KRBTEST.COM, Server not found in Kerberos database
Aug 12 06:55:45 dist-test-slave-65mx krb5kdc[29165](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.52.67@KRBTEST.COM for oryx/127.19.52.126@KRBTEST.COM, Server not found in Kerberos database
I20260812 06:55:45.469823 30328 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:45.458704 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.126:44739 (local address 127.19.52.67:33523)
0812 06:55:45.459104 (+ 400us) negotiation.cc:107] Waiting for socket to connect
0812 06:55:45.459127 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0812 06:55:45.459269 (+ 142us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:55:45.459662 (+ 393us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:55:45.459665 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:55:45.459931 (+ 266us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:55:45.460080 (+ 149us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:55:45.460087 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:55:45.460926 (+ 839us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:55:45.460929 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:55:45.461638 (+ 709us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:55:45.461649 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:55:45.462172 (+ 523us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:55:45.462175 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:55:45.462244 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:55:45.467515 (+ 5271us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 06:55:45.467531 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:55:45.469693 (+ 2162us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.19.52.126:44739: Server oryx/127.19.52.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":315,"thread_start_us":110,"threads_started":1}
W20260812 06:55:45.470191 30326 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 06:55:45.458848 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.52.67:33523 (local address 127.19.52.126:44739)
0812 06:55:45.458984 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0812 06:55:45.458990 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 06:55:45.459216 (+ 226us) server_negotiation.cc:407] Connection header received
0812 06:55:45.459429 (+ 213us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:55:45.459432 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:55:45.459491 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:55:45.459606 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:55:45.460207 (+ 601us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:45.460813 (+ 606us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:45.461782 (+ 969us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:45.462021 (+ 239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:45.462131 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:55:45.470035 (+ 7904us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.19.52.67:33523: BlockingRecv error: recv got EOF from 127.19.52.67:33523 (error 108)
Metrics: {"server-negotiator.queue_time_us":44}
Aug 12 06:55:46 dist-test-slave-65mx krb5kdc[29165](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.52.65@KRBTEST.COM for oryx/127.19.52.126@KRBTEST.COM, Server not found in Kerberos database
Aug 12 06:55:46 dist-test-slave-65mx krb5kdc[29165](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.52.65@KRBTEST.COM for oryx/127.19.52.126@KRBTEST.COM, Server not found in Kerberos database
I20260812 06:55:46.031544 30329 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:46.025016 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.126:44739 (local address 127.19.52.65:41309)
0812 06:55:46.025367 (+ 351us) negotiation.cc:107] Waiting for socket to connect
0812 06:55:46.025384 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0812 06:55:46.025555 (+ 171us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:55:46.025861 (+ 306us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:55:46.025864 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:55:46.026119 (+ 255us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:55:46.026220 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:55:46.026227 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:55:46.027028 (+ 801us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:55:46.027033 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:55:46.027600 (+ 567us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:55:46.027610 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:55:46.028100 (+ 490us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:55:46.028103 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:55:46.028191 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:55:46.028884 (+ 693us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 06:55:46.028896 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:55:46.031373 (+ 2477us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.19.52.126:44739: Server oryx/127.19.52.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":248,"thread_start_us":106,"threads_started":1}
W20260812 06:55:46.031708 30330 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 06:55:46.025209 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.52.65:41309 (local address 127.19.52.126:44739)
0812 06:55:46.025435 (+ 226us) server_negotiation.cc:206] Beginning negotiation
0812 06:55:46.025438 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 06:55:46.025518 (+ 80us) server_negotiation.cc:407] Connection header received
0812 06:55:46.025664 (+ 146us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:55:46.025667 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:55:46.025727 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:55:46.025833 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:55:46.026355 (+ 522us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:46.026909 (+ 554us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:46.027730 (+ 821us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:46.027972 (+ 242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:46.028084 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:55:46.031561 (+ 3477us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.19.52.65:41309: BlockingRecv error: recv got EOF from 127.19.52.65:41309 (error 108)
Metrics: {"server-negotiator.queue_time_us":151,"thread_start_us":64,"threads_started":1}
Aug 12 06:55:46 dist-test-slave-65mx krb5kdc[29165](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.52.66@KRBTEST.COM for oryx/127.19.52.126@KRBTEST.COM, Server not found in Kerberos database
Aug 12 06:55:46 dist-test-slave-65mx krb5kdc[29165](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.52.66@KRBTEST.COM for oryx/127.19.52.126@KRBTEST.COM, Server not found in Kerberos database
I20260812 06:55:46.255188 30331 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:46.248501 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.126:44739 (local address 127.19.52.66:44509)
0812 06:55:46.248868 (+ 367us) negotiation.cc:107] Waiting for socket to connect
0812 06:55:46.248882 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 06:55:46.248990 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:55:46.249264 (+ 274us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:55:46.249267 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:55:46.249489 (+ 222us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:55:46.249611 (+ 122us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:55:46.249617 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:55:46.250401 (+ 784us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:55:46.250404 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:55:46.251001 (+ 597us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:55:46.251010 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:55:46.251609 (+ 599us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:55:46.251612 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:55:46.251683 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:55:46.252435 (+ 752us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 06:55:46.252452 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:55:46.255027 (+ 2575us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.19.52.126:44739: Server oryx/127.19.52.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":283,"thread_start_us":123,"threads_started":1}
W20260812 06:55:46.255345 30330 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 06:55:46.248667 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.52.66:44509 (local address 127.19.52.126:44739)
0812 06:55:46.248858 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0812 06:55:46.248862 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 06:55:46.248990 (+ 128us) server_negotiation.cc:407] Connection header received
0812 06:55:46.249064 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:55:46.249066 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:55:46.249112 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:55:46.249187 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:55:46.249738 (+ 551us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:46.250283 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:46.251126 (+ 843us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:46.251374 (+ 248us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:46.251593 (+ 219us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:55:46.255210 (+ 3617us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.19.52.66:44509: BlockingRecv error: recv got EOF from 127.19.52.66:44509 (error 108)
Metrics: {"server-negotiator.queue_time_us":76}
Aug 12 06:55:46 dist-test-slave-65mx krb5kdc[29165](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.52.67@KRBTEST.COM for oryx/127.19.52.126@KRBTEST.COM, Server not found in Kerberos database
Aug 12 06:55:46 dist-test-slave-65mx krb5kdc[29165](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.52.67@KRBTEST.COM for oryx/127.19.52.126@KRBTEST.COM, Server not found in Kerberos database
I20260812 06:55:46.477865 30332 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:46.470745 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.126:44739 (local address 127.19.52.67:54773)
0812 06:55:46.471156 (+ 411us) negotiation.cc:107] Waiting for socket to connect
0812 06:55:46.471169 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 06:55:46.471272 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:55:46.471685 (+ 413us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:55:46.471687 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:55:46.471887 (+ 200us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:55:46.472033 (+ 146us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:55:46.472039 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:55:46.472956 (+ 917us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:55:46.472959 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:55:46.473649 (+ 690us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:55:46.473661 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:55:46.474184 (+ 523us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:55:46.474187 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:55:46.474261 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:55:46.474890 (+ 629us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 06:55:46.474902 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:55:46.477710 (+ 2808us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.19.52.126:44739: Server oryx/127.19.52.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":343,"thread_start_us":120,"threads_started":1}
W20260812 06:55:46.478124 30330 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 06:55:46.470881 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.52.67:54773 (local address 127.19.52.126:44739)
0812 06:55:46.471041 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0812 06:55:46.471044 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 06:55:46.471248 (+ 204us) server_negotiation.cc:407] Connection header received
0812 06:55:46.471354 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:55:46.471356 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:55:46.471408 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:55:46.471623 (+ 215us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:55:46.472165 (+ 542us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:46.472837 (+ 672us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:46.473781 (+ 944us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:46.474045 (+ 264us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:46.474146 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:55:46.477936 (+ 3790us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.19.52.67:54773: BlockingRecv error: recv got EOF from 127.19.52.67:54773 (error 108)
Metrics: {"server-negotiator.queue_time_us":66}
Aug 12 06:55:47 dist-test-slave-65mx krb5kdc[29165](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.52.65@KRBTEST.COM for oryx/127.19.52.126@KRBTEST.COM, Server not found in Kerberos database
Aug 12 06:55:47 dist-test-slave-65mx krb5kdc[29165](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.52.65@KRBTEST.COM for oryx/127.19.52.126@KRBTEST.COM, Server not found in Kerberos database
I20260812 06:55:47.038811 30333 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:47.032794 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.126:44739 (local address 127.19.52.65:39851)
0812 06:55:47.033144 (+ 350us) negotiation.cc:107] Waiting for socket to connect
0812 06:55:47.033159 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 06:55:47.033271 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:55:47.033575 (+ 304us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:55:47.033578 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:55:47.033790 (+ 212us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:55:47.033898 (+ 108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:55:47.033904 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:55:47.034830 (+ 926us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:55:47.034835 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:55:47.035443 (+ 608us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:55:47.035453 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:55:47.035965 (+ 512us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:55:47.035969 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:55:47.036060 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:55:47.036659 (+ 599us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 06:55:47.036672 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:55:47.038672 (+ 2000us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.19.52.126:44739: Server oryx/127.19.52.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":257,"thread_start_us":96,"threads_started":1}
W20260812 06:55:47.039113 30334 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 06:55:47.032883 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.52.65:39851 (local address 127.19.52.126:44739)
0812 06:55:47.033160 (+ 277us) server_negotiation.cc:206] Beginning negotiation
0812 06:55:47.033164 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 06:55:47.033249 (+ 85us) server_negotiation.cc:407] Connection header received
0812 06:55:47.033365 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:55:47.033373 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:55:47.033421 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:55:47.033529 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:55:47.034040 (+ 511us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:47.034708 (+ 668us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:47.035585 (+ 877us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:47.035840 (+ 255us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:47.035958 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:55:47.038894 (+ 2936us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.19.52.65:39851: BlockingRecv error: recv got EOF from 127.19.52.65:39851 (error 108)
Metrics: {"server-negotiator.queue_time_us":192,"thread_start_us":71,"threads_started":1}
Aug 12 06:55:47 dist-test-slave-65mx krb5kdc[29165](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.52.66@KRBTEST.COM for oryx/127.19.52.126@KRBTEST.COM, Server not found in Kerberos database
Aug 12 06:55:47 dist-test-slave-65mx krb5kdc[29165](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.52.66@KRBTEST.COM for oryx/127.19.52.126@KRBTEST.COM, Server not found in Kerberos database
I20260812 06:55:47.262764 30335 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:47.256270 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.126:44739 (local address 127.19.52.66:48173)
0812 06:55:47.256703 (+ 433us) negotiation.cc:107] Waiting for socket to connect
0812 06:55:47.256727 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0812 06:55:47.256879 (+ 152us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:55:47.257292 (+ 413us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:55:47.257295 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:55:47.257544 (+ 249us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:55:47.257648 (+ 104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:55:47.257654 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:55:47.258519 (+ 865us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:55:47.258525 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:55:47.259152 (+ 627us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:55:47.259161 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:55:47.259679 (+ 518us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:55:47.259683 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:55:47.259779 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:55:47.260430 (+ 651us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 06:55:47.260443 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:55:47.262563 (+ 2120us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.19.52.126:44739: Server oryx/127.19.52.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":340,"thread_start_us":93,"threads_started":1}
W20260812 06:55:47.262857 30334 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 06:55:47.256408 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.52.66:48173 (local address 127.19.52.126:44739)
0812 06:55:47.256666 (+ 258us) server_negotiation.cc:206] Beginning negotiation
0812 06:55:47.256671 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 06:55:47.256818 (+ 147us) server_negotiation.cc:407] Connection header received
0812 06:55:47.257100 (+ 282us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:55:47.257104 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:55:47.257165 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:55:47.257269 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:55:47.257786 (+ 517us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:47.258381 (+ 595us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:47.259288 (+ 907us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:47.259544 (+ 256us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:47.259671 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:55:47.262726 (+ 3055us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.19.52.66:48173: BlockingRecv error: recv got EOF from 127.19.52.66:48173 (error 108)
Metrics: {"server-negotiator.queue_time_us":99}
Aug 12 06:55:47 dist-test-slave-65mx krb5kdc[29165](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.52.67@KRBTEST.COM for oryx/127.19.52.126@KRBTEST.COM, Server not found in Kerberos database
Aug 12 06:55:47 dist-test-slave-65mx krb5kdc[29165](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.52.67@KRBTEST.COM for oryx/127.19.52.126@KRBTEST.COM, Server not found in Kerberos database
I20260812 06:55:47.485796 30336 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:47.478933 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.126:44739 (local address 127.19.52.67:37307)
0812 06:55:47.479218 (+ 285us) negotiation.cc:107] Waiting for socket to connect
0812 06:55:47.479238 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0812 06:55:47.479353 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:55:47.479785 (+ 432us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:55:47.479787 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:55:47.480189 (+ 402us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:55:47.480306 (+ 117us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:55:47.480314 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:55:47.481329 (+ 1015us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:55:47.481335 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:55:47.482136 (+ 801us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:55:47.482147 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:55:47.482678 (+ 531us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:55:47.482682 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:55:47.482780 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:55:47.483581 (+ 801us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 06:55:47.483594 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:55:47.485670 (+ 2076us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.19.52.126:44739: Server oryx/127.19.52.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":194,"thread_start_us":98,"threads_started":1}
W20260812 06:55:47.486068 30334 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 06:55:47.479166 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.52.67:37307 (local address 127.19.52.126:44739)
0812 06:55:47.479311 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0812 06:55:47.479316 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 06:55:47.479331 (+ 15us) server_negotiation.cc:407] Connection header received
0812 06:55:47.479597 (+ 266us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:55:47.479601 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:55:47.479668 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:55:47.479759 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:55:47.480588 (+ 829us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:47.481191 (+ 603us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:47.482283 (+ 1092us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:47.482540 (+ 257us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:47.482670 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:55:47.485894 (+ 3224us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.19.52.67:37307: BlockingRecv error: recv got EOF from 127.19.52.67:37307 (error 108)
Metrics: {"server-negotiator.queue_time_us":42}
Aug 12 06:55:48 dist-test-slave-65mx krb5kdc[29165](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.52.65@KRBTEST.COM for oryx/127.19.52.126@KRBTEST.COM, Server not found in Kerberos database
Aug 12 06:55:48 dist-test-slave-65mx krb5kdc[29165](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.52.65@KRBTEST.COM for oryx/127.19.52.126@KRBTEST.COM, Server not found in Kerberos database
I20260812 06:55:48.046311 30337 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:48.039935 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.126:44739 (local address 127.19.52.65:42937)
0812 06:55:48.040282 (+ 347us) negotiation.cc:107] Waiting for socket to connect
0812 06:55:48.040296 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 06:55:48.040426 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:55:48.040719 (+ 293us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:55:48.040721 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:55:48.040912 (+ 191us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:55:48.041071 (+ 159us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:55:48.041077 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:55:48.041908 (+ 831us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:55:48.041912 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:55:48.042731 (+ 819us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:55:48.042744 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:55:48.043292 (+ 548us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:55:48.043296 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:55:48.043372 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:55:48.043974 (+ 602us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 06:55:48.043989 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:55:48.046178 (+ 2189us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.19.52.126:44739: Server oryx/127.19.52.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":257,"thread_start_us":139,"threads_started":1}
W20260812 06:55:48.046535 30338 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 06:55:48.040020 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.52.65:42937 (local address 127.19.52.126:44739)
0812 06:55:48.040299 (+ 279us) server_negotiation.cc:206] Beginning negotiation
0812 06:55:48.040302 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 06:55:48.040399 (+ 97us) server_negotiation.cc:407] Connection header received
0812 06:55:48.040524 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:55:48.040526 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:55:48.040589 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:55:48.040672 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:55:48.041186 (+ 514us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:48.041746 (+ 560us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:48.042848 (+ 1102us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:48.043168 (+ 320us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:48.043273 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:55:48.046350 (+ 3077us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.19.52.65:42937: BlockingRecv error: recv got EOF from 127.19.52.65:42937 (error 108)
Metrics: {"server-negotiator.queue_time_us":187,"thread_start_us":70,"threads_started":1}
Aug 12 06:55:48 dist-test-slave-65mx krb5kdc[29165](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.52.66@KRBTEST.COM for oryx/127.19.52.126@KRBTEST.COM, Server not found in Kerberos database
Aug 12 06:55:48 dist-test-slave-65mx krb5kdc[29165](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.52.66@KRBTEST.COM for oryx/127.19.52.126@KRBTEST.COM, Server not found in Kerberos database
I20260812 06:55:48.270471 30339 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:48.263960 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.126:44739 (local address 127.19.52.66:42975)
0812 06:55:48.264253 (+ 293us) negotiation.cc:107] Waiting for socket to connect
0812 06:55:48.264272 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0812 06:55:48.264376 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:55:48.264752 (+ 376us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:55:48.264755 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:55:48.264972 (+ 217us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:55:48.265079 (+ 107us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:55:48.265085 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:55:48.265890 (+ 805us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:55:48.265893 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:55:48.266490 (+ 597us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:55:48.266518 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:55:48.267069 (+ 551us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:55:48.267072 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:55:48.267145 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:55:48.267786 (+ 641us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 06:55:48.267799 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:55:48.270328 (+ 2529us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.19.52.126:44739: Server oryx/127.19.52.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":208,"thread_start_us":101,"threads_started":1}
W20260812 06:55:48.270745 30338 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 06:55:48.264030 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.52.66:42975 (local address 127.19.52.126:44739)
0812 06:55:48.264225 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0812 06:55:48.264229 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 06:55:48.264345 (+ 116us) server_negotiation.cc:407] Connection header received
0812 06:55:48.264513 (+ 168us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:55:48.264517 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:55:48.264590 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:55:48.264670 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:55:48.265199 (+ 529us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:48.265754 (+ 555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:48.266660 (+ 906us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:48.266938 (+ 278us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:48.267022 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:55:48.270619 (+ 3597us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.19.52.66:42975: BlockingRecv error: recv got EOF from 127.19.52.66:42975 (error 108)
Metrics: {"server-negotiator.queue_time_us":95}
Aug 12 06:55:48 dist-test-slave-65mx krb5kdc[29165](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.52.67@KRBTEST.COM for oryx/127.19.52.126@KRBTEST.COM, Server not found in Kerberos database
Aug 12 06:55:48 dist-test-slave-65mx krb5kdc[29165](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.52.67@KRBTEST.COM for oryx/127.19.52.126@KRBTEST.COM, Server not found in Kerberos database
I20260812 06:55:48.493126 30340 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:48.486837 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.126:44739 (local address 127.19.52.67:37079)
0812 06:55:48.487159 (+ 322us) negotiation.cc:107] Waiting for socket to connect
0812 06:55:48.487178 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0812 06:55:48.487285 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:55:48.487570 (+ 285us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:55:48.487573 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:55:48.487768 (+ 195us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:55:48.487865 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:55:48.487870 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:55:48.488738 (+ 868us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:55:48.488741 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:55:48.489341 (+ 600us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:55:48.489350 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:55:48.489907 (+ 557us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:55:48.489914 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:55:48.490075 (+ 161us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:55:48.490723 (+ 648us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 06:55:48.490736 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:55:48.492938 (+ 2202us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.19.52.126:44739: Server oryx/127.19.52.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":193,"thread_start_us":106,"threads_started":1}
W20260812 06:55:48.493230 30338 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 06:55:48.487003 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.52.67:37079 (local address 127.19.52.126:44739)
0812 06:55:48.487147 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0812 06:55:48.487150 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 06:55:48.487299 (+ 149us) server_negotiation.cc:407] Connection header received
0812 06:55:48.487360 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:55:48.487362 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:55:48.487413 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:55:48.487497 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:55:48.488049 (+ 552us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:48.488606 (+ 557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:48.489469 (+ 863us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:48.489702 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:48.489787 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:55:48.493089 (+ 3302us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.19.52.67:37079: BlockingRecv error: recv got EOF from 127.19.52.67:37079 (error 108)
Metrics: {"server-negotiator.queue_time_us":60}
Aug 12 06:55:49 dist-test-slave-65mx krb5kdc[29165](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.52.65@KRBTEST.COM for oryx/127.19.52.126@KRBTEST.COM, Server not found in Kerberos database
Aug 12 06:55:49 dist-test-slave-65mx krb5kdc[29165](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.52.65@KRBTEST.COM for oryx/127.19.52.126@KRBTEST.COM, Server not found in Kerberos database
I20260812 06:55:49.053474 30341 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:49.047223 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.126:44739 (local address 127.19.52.65:37405)
0812 06:55:49.047519 (+ 296us) negotiation.cc:107] Waiting for socket to connect
0812 06:55:49.047540 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0812 06:55:49.047634 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:55:49.047921 (+ 287us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:55:49.047924 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:55:49.048169 (+ 245us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:55:49.048273 (+ 104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:55:49.048279 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:55:49.049153 (+ 874us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:55:49.049156 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:55:49.049738 (+ 582us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:55:49.049748 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:55:49.050222 (+ 474us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:55:49.050225 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:55:49.050293 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:55:49.050922 (+ 629us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 06:55:49.050979 (+ 57us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:55:49.053283 (+ 2304us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.19.52.126:44739: Server oryx/127.19.52.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":208,"thread_start_us":108,"threads_started":1}
W20260812 06:55:49.053573 30342 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 06:55:49.047325 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.52.65:37405 (local address 127.19.52.126:44739)
0812 06:55:49.047587 (+ 262us) server_negotiation.cc:206] Beginning negotiation
0812 06:55:49.047590 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 06:55:49.047604 (+ 14us) server_negotiation.cc:407] Connection header received
0812 06:55:49.047721 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:55:49.047723 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:55:49.047767 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:55:49.047851 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:55:49.048425 (+ 574us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:49.049011 (+ 586us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:49.049860 (+ 849us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:49.050100 (+ 240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:49.050185 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:55:49.053435 (+ 3250us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.19.52.65:37405: BlockingRecv error: recv got EOF from 127.19.52.65:37405 (error 108)
Metrics: {"server-negotiator.queue_time_us":193,"thread_start_us":85,"threads_started":1}
Aug 12 06:55:49 dist-test-slave-65mx krb5kdc[29165](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.52.66@KRBTEST.COM for oryx/127.19.52.126@KRBTEST.COM, Server not found in Kerberos database
Aug 12 06:55:49 dist-test-slave-65mx krb5kdc[29165](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.52.66@KRBTEST.COM for oryx/127.19.52.126@KRBTEST.COM, Server not found in Kerberos database
I20260812 06:55:49.277736 30344 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:49.271443 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.126:44739 (local address 127.19.52.66:40115)
0812 06:55:49.271863 (+ 420us) negotiation.cc:107] Waiting for socket to connect
0812 06:55:49.271879 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0812 06:55:49.272041 (+ 162us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:55:49.272353 (+ 312us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:55:49.272356 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:55:49.272686 (+ 330us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:55:49.272792 (+ 106us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:55:49.272798 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:55:49.273627 (+ 829us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:55:49.273633 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:55:49.274206 (+ 573us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:55:49.274215 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:55:49.274708 (+ 493us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:55:49.274712 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:55:49.274801 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:55:49.275403 (+ 602us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 06:55:49.275424 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:55:49.277560 (+ 2136us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.19.52.126:44739: Server oryx/127.19.52.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":333,"thread_start_us":113,"threads_started":1}
W20260812 06:55:49.277841 30342 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 06:55:49.271622 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.52.66:40115 (local address 127.19.52.126:44739)
0812 06:55:49.271828 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0812 06:55:49.271833 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 06:55:49.271995 (+ 162us) server_negotiation.cc:407] Connection header received
0812 06:55:49.272170 (+ 175us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:55:49.272176 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:55:49.272231 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:55:49.272325 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:55:49.272939 (+ 614us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:49.273500 (+ 561us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:49.274337 (+ 837us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:49.274577 (+ 240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:49.274689 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:55:49.277713 (+ 3024us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.19.52.66:40115: BlockingRecv error: recv got EOF from 127.19.52.66:40115 (error 108)
Metrics: {"server-negotiator.queue_time_us":84}
Aug 12 06:55:49 dist-test-slave-65mx krb5kdc[29165](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.52.67@KRBTEST.COM for oryx/127.19.52.126@KRBTEST.COM, Server not found in Kerberos database
Aug 12 06:55:49 dist-test-slave-65mx krb5kdc[29165](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.52.67@KRBTEST.COM for oryx/127.19.52.126@KRBTEST.COM, Server not found in Kerberos database
I20260812 06:55:49.500243 30345 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:49.494249 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.126:44739 (local address 127.19.52.67:47969)
0812 06:55:49.494611 (+ 362us) negotiation.cc:107] Waiting for socket to connect
0812 06:55:49.494625 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 06:55:49.494774 (+ 149us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:55:49.495116 (+ 342us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:55:49.495120 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:55:49.495339 (+ 219us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:55:49.495434 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:55:49.495439 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:55:49.496343 (+ 904us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:55:49.496348 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:55:49.496986 (+ 638us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:55:49.496995 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:55:49.497476 (+ 481us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:55:49.497480 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:55:49.497572 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:55:49.498123 (+ 551us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 06:55:49.498135 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:55:49.500116 (+ 1981us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.19.52.126:44739: Server oryx/127.19.52.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":264,"thread_start_us":126,"threads_started":1}
W20260812 06:55:49.500469 30342 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 06:55:49.494333 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.52.67:47969 (local address 127.19.52.126:44739)
0812 06:55:49.494530 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0812 06:55:49.494535 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 06:55:49.494706 (+ 171us) server_negotiation.cc:407] Connection header received
0812 06:55:49.494903 (+ 197us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:55:49.494906 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:55:49.494953 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:55:49.495044 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:55:49.495596 (+ 552us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:49.496227 (+ 631us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:49.497129 (+ 902us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:49.497356 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:49.497467 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:55:49.500333 (+ 2866us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.19.52.67:47969: BlockingRecv error: recv got EOF from 127.19.52.67:47969 (error 108)
Metrics: {"server-negotiator.queue_time_us":52}
Aug 12 06:55:50 dist-test-slave-65mx krb5kdc[29165](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.52.65@KRBTEST.COM for oryx/127.19.52.126@KRBTEST.COM, Server not found in Kerberos database
Aug 12 06:55:50 dist-test-slave-65mx krb5kdc[29165](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.52.65@KRBTEST.COM for oryx/127.19.52.126@KRBTEST.COM, Server not found in Kerberos database
I20260812 06:55:50.061180 30346 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:50.054692 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.126:44739 (local address 127.19.52.65:57603)
0812 06:55:50.055069 (+ 377us) negotiation.cc:107] Waiting for socket to connect
0812 06:55:50.055083 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 06:55:50.055187 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:55:50.055693 (+ 506us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:55:50.055697 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:55:50.055934 (+ 237us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:55:50.056038 (+ 104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:55:50.056044 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:55:50.057043 (+ 999us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:55:50.057046 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:55:50.057686 (+ 640us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:55:50.057697 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:55:50.058218 (+ 521us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:55:50.058221 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:55:50.058285 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:55:50.058968 (+ 683us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 06:55:50.058982 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:55:50.061032 (+ 2050us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.19.52.126:44739: Server oryx/127.19.52.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":271,"thread_start_us":134,"threads_started":1}
W20260812 06:55:50.061447 30348 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 06:55:50.054802 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.52.65:57603 (local address 127.19.52.126:44739)
0812 06:55:50.055130 (+ 328us) server_negotiation.cc:206] Beginning negotiation
0812 06:55:50.055134 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 06:55:50.055149 (+ 15us) server_negotiation.cc:407] Connection header received
0812 06:55:50.055453 (+ 304us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:55:50.055456 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:55:50.055520 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:55:50.055666 (+ 146us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:55:50.056211 (+ 545us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:50.056874 (+ 663us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:50.057832 (+ 958us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:50.058101 (+ 269us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:50.058192 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:55:50.061277 (+ 3085us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.19.52.65:57603: BlockingRecv error: recv got EOF from 127.19.52.65:57603 (error 108)
Metrics: {"server-negotiator.queue_time_us":239,"thread_start_us":100,"threads_started":1}
Aug 12 06:55:50 dist-test-slave-65mx krb5kdc[29165](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.52.66@KRBTEST.COM for oryx/127.19.52.126@KRBTEST.COM, Server not found in Kerberos database
Aug 12 06:55:50 dist-test-slave-65mx krb5kdc[29165](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.52.66@KRBTEST.COM for oryx/127.19.52.126@KRBTEST.COM, Server not found in Kerberos database
I20260812 06:55:50.284801 30349 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:50.278752 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.126:44739 (local address 127.19.52.66:35361)
0812 06:55:50.279067 (+ 315us) negotiation.cc:107] Waiting for socket to connect
0812 06:55:50.279086 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0812 06:55:50.279201 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:55:50.279481 (+ 280us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:55:50.279483 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:55:50.279676 (+ 193us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:55:50.279773 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:55:50.279778 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:55:50.280654 (+ 876us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:55:50.280663 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:55:50.281230 (+ 567us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:55:50.281238 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:55:50.281674 (+ 436us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:55:50.281677 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:55:50.281742 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:55:50.282311 (+ 569us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 06:55:50.282323 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:55:50.284580 (+ 2257us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.19.52.126:44739: Server oryx/127.19.52.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":229,"thread_start_us":132,"threads_started":1}
W20260812 06:55:50.284938 30348 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 06:55:50.278958 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.52.66:35361 (local address 127.19.52.126:44739)
0812 06:55:50.279126 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0812 06:55:50.279130 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 06:55:50.279150 (+ 20us) server_negotiation.cc:407] Connection header received
0812 06:55:50.279296 (+ 146us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:55:50.279298 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:55:50.279343 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:55:50.279423 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:55:50.279913 (+ 490us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:50.280468 (+ 555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:50.281343 (+ 875us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:50.281560 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:50.281645 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:55:50.284816 (+ 3171us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.19.52.66:35361: BlockingRecv error: recv got EOF from 127.19.52.66:35361 (error 108)
Metrics: {"server-negotiator.queue_time_us":60}
Aug 12 06:55:50 dist-test-slave-65mx krb5kdc[29165](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.52.67@KRBTEST.COM for oryx/127.19.52.126@KRBTEST.COM, Server not found in Kerberos database
Aug 12 06:55:50 dist-test-slave-65mx krb5kdc[29165](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.52.67@KRBTEST.COM for oryx/127.19.52.126@KRBTEST.COM, Server not found in Kerberos database
I20260812 06:55:50.511204 30350 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:50.501276 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.126:44739 (local address 127.19.52.67:55953)
0812 06:55:50.501552 (+ 276us) negotiation.cc:107] Waiting for socket to connect
0812 06:55:50.501566 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 06:55:50.501699 (+ 133us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:55:50.502027 (+ 328us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:55:50.502029 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:55:50.502215 (+ 186us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:55:50.502313 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:55:50.502318 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:55:50.503070 (+ 752us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:55:50.503072 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:55:50.504006 (+ 934us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:55:50.504020 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:55:50.504690 (+ 670us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:55:50.504694 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:55:50.504781 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:55:50.508916 (+ 4135us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 06:55:50.508932 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:55:50.511061 (+ 2129us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.19.52.126:44739: Server oryx/127.19.52.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":199,"thread_start_us":78,"threads_started":1}
W20260812 06:55:50.511446 30348 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 06:55:50.501368 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.52.67:55953 (local address 127.19.52.126:44739)
0812 06:55:50.501565 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0812 06:55:50.501569 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 06:55:50.501625 (+ 56us) server_negotiation.cc:407] Connection header received
0812 06:55:50.501832 (+ 207us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:55:50.501836 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:55:50.501900 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:55:50.501988 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:55:50.502449 (+ 461us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:50.502973 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:50.504125 (+ 1152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:50.504512 (+ 387us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:50.504675 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:55:50.511305 (+ 6630us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.19.52.67:55953: BlockingRecv error: recv got EOF from 127.19.52.67:55953 (error 108)
Metrics: {"server-negotiator.queue_time_us":101}
Aug 12 06:55:51 dist-test-slave-65mx krb5kdc[29165](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.52.65@KRBTEST.COM for oryx/127.19.52.126@KRBTEST.COM, Server not found in Kerberos database
Aug 12 06:55:51 dist-test-slave-65mx krb5kdc[29165](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.52.65@KRBTEST.COM for oryx/127.19.52.126@KRBTEST.COM, Server not found in Kerberos database
I20260812 06:55:51.069339 30351 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:51.062239 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.126:44739 (local address 127.19.52.65:39119)
0812 06:55:51.062655 (+ 416us) negotiation.cc:107] Waiting for socket to connect
0812 06:55:51.062693 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0812 06:55:51.062827 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:55:51.063188 (+ 361us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:55:51.063192 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:55:51.063470 (+ 278us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:55:51.063577 (+ 107us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:55:51.063584 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:55:51.064472 (+ 888us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:55:51.064478 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:55:51.065196 (+ 718us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:55:51.065207 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:55:51.065753 (+ 546us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:55:51.065757 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:55:51.065856 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:55:51.066553 (+ 697us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 06:55:51.066567 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:55:51.069146 (+ 2579us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.19.52.126:44739: Server oryx/127.19.52.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":328,"thread_start_us":109,"threads_started":1}
W20260812 06:55:51.069451 30353 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 06:55:51.062308 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.52.65:39119 (local address 127.19.52.126:44739)
0812 06:55:51.062669 (+ 361us) server_negotiation.cc:206] Beginning negotiation
0812 06:55:51.062674 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 06:55:51.062782 (+ 108us) server_negotiation.cc:407] Connection header received
0812 06:55:51.062948 (+ 166us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:55:51.062954 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:55:51.063021 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:55:51.063153 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:55:51.063723 (+ 570us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:51.064329 (+ 606us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:51.065341 (+ 1012us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:51.065613 (+ 272us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:51.065744 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:55:51.069307 (+ 3563us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.19.52.65:39119: BlockingRecv error: recv got EOF from 127.19.52.65:39119 (error 108)
Metrics: {"server-negotiator.queue_time_us":275,"thread_start_us":99,"threads_started":1}
Aug 12 06:55:51 dist-test-slave-65mx krb5kdc[29165](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.52.66@KRBTEST.COM for oryx/127.19.52.126@KRBTEST.COM, Server not found in Kerberos database
Aug 12 06:55:51 dist-test-slave-65mx krb5kdc[29165](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.52.66@KRBTEST.COM for oryx/127.19.52.126@KRBTEST.COM, Server not found in Kerberos database
I20260812 06:55:51.292112 30354 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:51.285750 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.126:44739 (local address 127.19.52.66:46103)
0812 06:55:51.286068 (+ 318us) negotiation.cc:107] Waiting for socket to connect
0812 06:55:51.286086 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0812 06:55:51.286205 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:55:51.286547 (+ 342us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:55:51.286550 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:55:51.286744 (+ 194us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:55:51.286844 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:55:51.286849 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:55:51.287822 (+ 973us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:55:51.287825 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:55:51.288486 (+ 661us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:55:51.288496 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:55:51.289129 (+ 633us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:55:51.289133 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:55:51.289231 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:55:51.289913 (+ 682us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 06:55:51.289926 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:55:51.291972 (+ 2046us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.19.52.126:44739: Server oryx/127.19.52.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":236,"thread_start_us":81,"threads_started":1}
W20260812 06:55:51.292299 30353 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 06:55:51.285839 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.52.66:46103 (local address 127.19.52.126:44739)
0812 06:55:51.286043 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0812 06:55:51.286047 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 06:55:51.286167 (+ 120us) server_negotiation.cc:407] Connection header received
0812 06:55:51.286301 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:55:51.286303 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:55:51.286349 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:55:51.286418 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:55:51.286957 (+ 539us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:51.287679 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:51.288724 (+ 1045us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:51.288987 (+ 263us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:51.289119 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:55:51.292163 (+ 3044us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.19.52.66:46103: BlockingRecv error: recv got EOF from 127.19.52.66:46103 (error 108)
Metrics: {"server-negotiator.queue_time_us":97}
Aug 12 06:55:51 dist-test-slave-65mx krb5kdc[29165](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.52.67@KRBTEST.COM for oryx/127.19.52.126@KRBTEST.COM, Server not found in Kerberos database
Aug 12 06:55:51 dist-test-slave-65mx krb5kdc[29165](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.52.67@KRBTEST.COM for oryx/127.19.52.126@KRBTEST.COM, Server not found in Kerberos database
I20260812 06:55:51.517900 30355 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:51.512128 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.126:44739 (local address 127.19.52.67:60963)
0812 06:55:51.512390 (+ 262us) negotiation.cc:107] Waiting for socket to connect
0812 06:55:51.512407 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0812 06:55:51.512595 (+ 188us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:55:51.512940 (+ 345us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:55:51.512943 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:55:51.513158 (+ 215us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:55:51.513259 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:55:51.513265 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:55:51.514047 (+ 782us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:55:51.514049 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:55:51.514677 (+ 628us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:55:51.514687 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:55:51.515120 (+ 433us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:55:51.515123 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:55:51.515196 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:55:51.515709 (+ 513us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 06:55:51.515725 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:55:51.517765 (+ 2040us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.19.52.126:44739: Server oryx/127.19.52.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":169,"thread_start_us":87,"threads_started":1}
W20260812 06:55:51.518165 30353 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 06:55:51.512175 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.52.67:60963 (local address 127.19.52.126:44739)
0812 06:55:51.512322 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0812 06:55:51.512327 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 06:55:51.512530 (+ 203us) server_negotiation.cc:407] Connection header received
0812 06:55:51.512703 (+ 173us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:55:51.512706 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:55:51.512791 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:55:51.512891 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:55:51.513390 (+ 499us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:51.513933 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:51.514789 (+ 856us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:51.515005 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:51.515087 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:55:51.517998 (+ 2911us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.19.52.67:60963: BlockingRecv error: recv got EOF from 127.19.52.67:60963 (error 108)
Metrics: {"server-negotiator.queue_time_us":56}
Aug 12 06:55:52 dist-test-slave-65mx krb5kdc[29165](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.52.65@KRBTEST.COM for oryx/127.19.52.126@KRBTEST.COM, Server not found in Kerberos database
Aug 12 06:55:52 dist-test-slave-65mx krb5kdc[29165](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.52.65@KRBTEST.COM for oryx/127.19.52.126@KRBTEST.COM, Server not found in Kerberos database
I20260812 06:55:52.076886 30356 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:52.070487 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.126:44739 (local address 127.19.52.65:33483)
0812 06:55:52.070834 (+ 347us) negotiation.cc:107] Waiting for socket to connect
0812 06:55:52.070861 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0812 06:55:52.070977 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:55:52.071253 (+ 276us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:55:52.071256 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:55:52.071465 (+ 209us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:55:52.071574 (+ 109us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:55:52.071580 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:55:52.072523 (+ 943us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:55:52.072525 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:55:52.073192 (+ 667us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:55:52.073203 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:55:52.073835 (+ 632us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:55:52.073839 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:55:52.073938 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:55:52.074655 (+ 717us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 06:55:52.074679 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:55:52.076760 (+ 2081us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.19.52.126:44739: Server oryx/127.19.52.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":234,"thread_start_us":84,"threads_started":1}
W20260812 06:55:52.077132 30357 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 06:55:52.070659 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.52.65:33483 (local address 127.19.52.126:44739)
0812 06:55:52.070974 (+ 315us) server_negotiation.cc:206] Beginning negotiation
0812 06:55:52.070979 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 06:55:52.071000 (+ 21us) server_negotiation.cc:407] Connection header received
0812 06:55:52.071052 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:55:52.071055 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:55:52.071111 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:55:52.071209 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:55:52.071743 (+ 534us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:52.072360 (+ 617us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:52.073455 (+ 1095us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:52.073691 (+ 236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:52.073821 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:55:52.076957 (+ 3136us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.19.52.65:33483: BlockingRecv error: recv got EOF from 127.19.52.65:33483 (error 108)
Metrics: {"server-negotiator.queue_time_us":223,"thread_start_us":89,"threads_started":1}
Aug 12 06:55:52 dist-test-slave-65mx krb5kdc[29165](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.52.66@KRBTEST.COM for oryx/127.19.52.126@KRBTEST.COM, Server not found in Kerberos database
Aug 12 06:55:52 dist-test-slave-65mx krb5kdc[29165](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.52.66@KRBTEST.COM for oryx/127.19.52.126@KRBTEST.COM, Server not found in Kerberos database
I20260812 06:55:52.301009 30358 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:52.293134 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.126:44739 (local address 127.19.52.66:38531)
0812 06:55:52.293592 (+ 458us) negotiation.cc:107] Waiting for socket to connect
0812 06:55:52.293608 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0812 06:55:52.293897 (+ 289us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:55:52.294283 (+ 386us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:55:52.294285 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:55:52.294468 (+ 183us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:55:52.294560 (+ 92us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:55:52.294565 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:55:52.295676 (+ 1111us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:55:52.295695 (+ 19us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:55:52.297207 (+ 1512us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:55:52.297224 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:55:52.297965 (+ 741us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:55:52.297969 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:55:52.298049 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:55:52.298634 (+ 585us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 06:55:52.298654 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:55:52.300814 (+ 2160us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.19.52.126:44739: Server oryx/127.19.52.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":369,"thread_start_us":91,"threads_started":1}
W20260812 06:55:52.301079 30357 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 06:55:52.293226 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.52.66:38531 (local address 127.19.52.126:44739)
0812 06:55:52.293416 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0812 06:55:52.293420 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 06:55:52.293688 (+ 268us) server_negotiation.cc:407] Connection header received
0812 06:55:52.294009 (+ 321us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:55:52.294012 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:55:52.294064 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:55:52.294139 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:55:52.294690 (+ 551us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:52.295550 (+ 860us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:52.297373 (+ 1823us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:52.297784 (+ 411us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:52.297917 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:55:52.300960 (+ 3043us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.19.52.66:38531: BlockingRecv error: recv got EOF from 127.19.52.66:38531 (error 108)
Metrics: {"server-negotiator.queue_time_us":71}
I20260812 06:55:52.377261 19665 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu with pid 29819
I20260812 06:55:52.386049 19665 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu with pid 29992
I20260812 06:55:52.398873 19665 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu with pid 30143
I20260812 06:55:52.407040 19665 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu with pid 29738
2026-08-12T06:55:52Z chronyd exiting
[ OK ] SecurityITest.TestMismatchingPrincipals (18381 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationInsecureCluster
2026-08-12T06:55:52Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T06:55:52Z Disabled control of system clock
I20260812 06:55:52.444155 19665 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu
/tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786517687384214-19665-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786517687384214-19665-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786517687384214-19665-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786517687384214-19665-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.52.126:44069
--webserver_interface=127.19.52.126
--webserver_port=0
--builtin_ntp_servers=127.19.52.84:44943
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.19.52.126:44069
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260812 06:55:52.564193 30362 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:55:52.564522 30362 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:55:52.564635 30362 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 06:55:52.568863 30362 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 06:55:52.568943 30362 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 06:55:52.568958 30362 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 06:55:52.568976 30362 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 06:55:52.568993 30362 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 06:55:52.574208 30362 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.52.84:44943
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786517687384214-19665-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786517687384214-19665-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.19.52.126:44069
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.52.126:44069
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786517687384214-19665-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.19.52.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30362
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786517687384214-19665-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 06:15:54 UTC on ccycloud.quasar-rnfzce.root.comops.site
build id 4581989
I20260812 06:55:52.575419 30362 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:55:52.576866 30362 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 06:55:52.586112 30370 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:55:52.586127 30367 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:55:52.586154 30368 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:55:52.586405 30362 server_base.cc:1034] running on GCE node
I20260812 06:55:52.587118 30362 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:55:52.588282 30362 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:55:52.589486 30362 hybrid_clock.cc:648] HybridClock initialized: now 1786517752589486 us; error 50 us; skew 500 ppm
I20260812 06:55:52.591562 30362 webserver.cc:460] Webserver started at http://127.19.52.126:34033/ using document root <none> and password file <none>
I20260812 06:55:52.592178 30362 fs_manager.cc:359] Metadata directory not provided
I20260812 06:55:52.592279 30362 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:55:52.592531 30362 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 06:55:52.594851 30362 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786517687384214-19665-0/minicluster-data/master-0/data/instance:
uuid: "ec011d59c9fa466997fbad0caf8863e8"
format_stamp: "Formatted at 2026-08-12 06:55:52 on dist-test-slave-65mx"
server_key: "36ec291424e06884b68e4c2d181dab4c"
server_key_iv: "aae58cafe4e9660c47e62b46ee6a7774"
server_key_version: "encryptionkey@0"
I20260812 06:55:52.595444 30362 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786517687384214-19665-0/minicluster-data/master-0/wal/instance:
uuid: "ec011d59c9fa466997fbad0caf8863e8"
format_stamp: "Formatted at 2026-08-12 06:55:52 on dist-test-slave-65mx"
server_key: "36ec291424e06884b68e4c2d181dab4c"
server_key_iv: "aae58cafe4e9660c47e62b46ee6a7774"
server_key_version: "encryptionkey@0"
I20260812 06:55:52.599417 30362 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.000s sys 0.006s
I20260812 06:55:52.601981 30377 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 06:55:52.603243 30362 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260812 06:55:52.603407 30362 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786517687384214-19665-0/minicluster-data/master-0/data,/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786517687384214-19665-0/minicluster-data/master-0/wal
uuid: "ec011d59c9fa466997fbad0caf8863e8"
format_stamp: "Formatted at 2026-08-12 06:55:52 on dist-test-slave-65mx"
server_key: "36ec291424e06884b68e4c2d181dab4c"
server_key_iv: "aae58cafe4e9660c47e62b46ee6a7774"
server_key_version: "encryptionkey@0"
I20260812 06:55:52.603669 30362 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786517687384214-19665-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786517687384214-19665-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786517687384214-19665-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:55:52.625036 30362 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 06:55:52.625854 30362 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 06:55:52.626104 30362 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:55:52.634734 30362 rpc_server.cc:307] RPC server started. Bound to: 127.19.52.126:44069
I20260812 06:55:52.634748 30434 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.52.126:44069 every 8 connection(s)
I20260812 06:55:52.635939 30362 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786517687384214-19665-0/minicluster-data/master-0/data/info.pb
I20260812 06:55:52.639678 30435 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 06:55:52.641496 19665 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu as pid 30362
I20260812 06:55:52.641599 19665 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786517687384214-19665-0/minicluster-data/master-0/wal/instance
I20260812 06:55:52.641932 19665 external_mini_cluster.cc:1428] Setting key 1cc6033e0eca42ae9ca4660732378166
I20260812 06:55:52.647053 30435 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ec011d59c9fa466997fbad0caf8863e8: Bootstrap starting.
I20260812 06:55:52.649621 30435 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ec011d59c9fa466997fbad0caf8863e8: Neither blocks nor log segments found. Creating new log.
I20260812 06:55:52.650492 30435 log.cc:824] T 00000000000000000000000000000000 P ec011d59c9fa466997fbad0caf8863e8: Log is configured to *not* fsync() on all Append() calls
I20260812 06:55:52.651948 30439 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:52.643606 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36652 (local address 127.19.52.126:44069)
0812 06:55:52.644354 (+ 748us) server_negotiation.cc:206] Beginning negotiation
0812 06:55:52.644371 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0812 06:55:52.644422 (+ 51us) server_negotiation.cc:407] Connection header received
0812 06:55:52.645213 (+ 791us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:55:52.645232 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:55:52.645669 (+ 437us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:55:52.646019 (+ 350us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:55:52.646466 (+ 447us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:52.647508 (+ 1042us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:52.648414 (+ 906us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:52.648970 (+ 556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:52.649159 (+ 189us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:55:52.650044 (+ 885us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:55:52.650050 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:55:52.650064 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 06:55:52.650474 (+ 410us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:55:52.650536 (+ 62us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 06:55:52.650557 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:55:52.650670 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0812 06:55:52.650812 (+ 142us) server_negotiation.cc:299] Negotiation successful
0812 06:55:52.651094 (+ 282us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":366,"thread_start_us":174,"threads_started":1}
I20260812 06:55:52.653281 30435 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ec011d59c9fa466997fbad0caf8863e8: No bootstrap required, opened a new log
I20260812 06:55:52.657305 30435 raft_consensus.cc:348] T 00000000000000000000000000000000 P ec011d59c9fa466997fbad0caf8863e8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ec011d59c9fa466997fbad0caf8863e8" member_type: VOTER last_known_addr { host: "127.19.52.126" port: 44069 } }
I20260812 06:55:52.657603 30435 raft_consensus.cc:374] T 00000000000000000000000000000000 P ec011d59c9fa466997fbad0caf8863e8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 06:55:52.657694 30435 raft_consensus.cc:729] T 00000000000000000000000000000000 P ec011d59c9fa466997fbad0caf8863e8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ec011d59c9fa466997fbad0caf8863e8, State: Initialized, Role: FOLLOWER
I20260812 06:55:52.658159 30435 consensus_queue.cc:260] T 00000000000000000000000000000000 P ec011d59c9fa466997fbad0caf8863e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ec011d59c9fa466997fbad0caf8863e8" member_type: VOTER last_known_addr { host: "127.19.52.126" port: 44069 } }
I20260812 06:55:52.658332 30435 raft_consensus.cc:388] T 00000000000000000000000000000000 P ec011d59c9fa466997fbad0caf8863e8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 06:55:52.658427 30435 raft_consensus.cc:482] T 00000000000000000000000000000000 P ec011d59c9fa466997fbad0caf8863e8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 06:55:52.658545 30435 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ec011d59c9fa466997fbad0caf8863e8 [term 0 FOLLOWER]: Advancing to term 1
I20260812 06:55:52.659869 30435 raft_consensus.cc:504] T 00000000000000000000000000000000 P ec011d59c9fa466997fbad0caf8863e8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ec011d59c9fa466997fbad0caf8863e8" member_type: VOTER last_known_addr { host: "127.19.52.126" port: 44069 } }
I20260812 06:55:52.660252 30435 leader_election.cc:302] T 00000000000000000000000000000000 P ec011d59c9fa466997fbad0caf8863e8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ec011d59c9fa466997fbad0caf8863e8; no voters:
I20260812 06:55:52.660656 30435 leader_election.cc:288] T 00000000000000000000000000000000 P ec011d59c9fa466997fbad0caf8863e8 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 06:55:52.660743 30442 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ec011d59c9fa466997fbad0caf8863e8 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 06:55:52.661051 30442 raft_consensus.cc:686] T 00000000000000000000000000000000 P ec011d59c9fa466997fbad0caf8863e8 [term 1 LEADER]: Becoming Leader. State: Replica: ec011d59c9fa466997fbad0caf8863e8, State: Running, Role: LEADER
I20260812 06:55:52.661614 30442 consensus_queue.cc:237] T 00000000000000000000000000000000 P ec011d59c9fa466997fbad0caf8863e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ec011d59c9fa466997fbad0caf8863e8" member_type: VOTER last_known_addr { host: "127.19.52.126" port: 44069 } }
I20260812 06:55:52.662088 30435 sys_catalog.cc:564] T 00000000000000000000000000000000 P ec011d59c9fa466997fbad0caf8863e8 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 06:55:52.663839 30444 sys_catalog.cc:455] T 00000000000000000000000000000000 P ec011d59c9fa466997fbad0caf8863e8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ec011d59c9fa466997fbad0caf8863e8. Latest consensus state: current_term: 1 leader_uuid: "ec011d59c9fa466997fbad0caf8863e8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ec011d59c9fa466997fbad0caf8863e8" member_type: VOTER last_known_addr { host: "127.19.52.126" port: 44069 } } }
I20260812 06:55:52.663892 30443 sys_catalog.cc:455] T 00000000000000000000000000000000 P ec011d59c9fa466997fbad0caf8863e8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ec011d59c9fa466997fbad0caf8863e8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ec011d59c9fa466997fbad0caf8863e8" member_type: VOTER last_known_addr { host: "127.19.52.126" port: 44069 } } }
I20260812 06:55:52.663991 30444 sys_catalog.cc:458] T 00000000000000000000000000000000 P ec011d59c9fa466997fbad0caf8863e8 [sys.catalog]: This master's current role is: LEADER
I20260812 06:55:52.664026 30443 sys_catalog.cc:458] T 00000000000000000000000000000000 P ec011d59c9fa466997fbad0caf8863e8 [sys.catalog]: This master's current role is: LEADER
I20260812 06:55:52.664408 30458 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 06:55:52.667809 30458 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 06:55:52.674688 30458 catalog_manager.cc:1347] Generated new cluster ID: 49a44e6d9a0e4c9d9fce4016e1687626
I20260812 06:55:52.674791 30458 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 06:55:52.691321 30458 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 06:55:52.692636 30458 catalog_manager.cc:1504] Loading token signing keys...
I20260812 06:55:52.710454 30458 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ec011d59c9fa466997fbad0caf8863e8: Generated new TSK 0
I20260812 06:55:52.711437 30458 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 06:55:52.724874 19665 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu
/tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786517687384214-19665-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786517687384214-19665-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786517687384214-19665-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786517687384214-19665-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.52.65:0
--local_ip_for_outbound_sockets=127.19.52.65
--webserver_interface=127.19.52.65
--webserver_port=0
--tserver_master_addrs=127.19.52.126:44069
--builtin_ntp_servers=127.19.52.84:44943
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260812 06:55:52.846114 30465 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:55:52.846393 30465 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:55:52.846433 30465 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 06:55:52.850456 30465 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 06:55:52.850524 30465 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 06:55:52.850581 30465 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.52.65
I20260812 06:55:52.855536 30465 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.52.84:44943
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786517687384214-19665-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786517687384214-19665-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.52.65:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786517687384214-19665-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.19.52.65
--webserver_port=0
--tserver_master_addrs=127.19.52.126:44069
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30465
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.52.65
--log_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786517687384214-19665-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 06:15:54 UTC on ccycloud.quasar-rnfzce.root.comops.site
build id 4581989
I20260812 06:55:52.856741 30465 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:55:52.858166 30465 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 06:55:52.868953 30471 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:55:52.868996 30472 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:55:52.868953 30474 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:55:52.869236 30465 server_base.cc:1034] running on GCE node
I20260812 06:55:52.869941 30465 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:55:52.870800 30465 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:55:52.872033 30465 hybrid_clock.cc:648] HybridClock initialized: now 1786517752871986 us; error 57 us; skew 500 ppm
I20260812 06:55:52.874322 30465 webserver.cc:460] Webserver started at http://127.19.52.65:45101/ using document root <none> and password file <none>
I20260812 06:55:52.875013 30465 fs_manager.cc:359] Metadata directory not provided
I20260812 06:55:52.875123 30465 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:55:52.875391 30465 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 06:55:52.878012 30465 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786517687384214-19665-0/minicluster-data/ts-0/data/instance:
uuid: "34c798dc3ce04e9e93c80db51effaebc"
format_stamp: "Formatted at 2026-08-12 06:55:52 on dist-test-slave-65mx"
server_key: "08716a575497f06922923357e9a535b0"
server_key_iv: "d356950f662fd4ad7b0b1ec809dc7572"
server_key_version: "encryptionkey@0"
I20260812 06:55:52.878760 30465 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786517687384214-19665-0/minicluster-data/ts-0/wal/instance:
uuid: "34c798dc3ce04e9e93c80db51effaebc"
format_stamp: "Formatted at 2026-08-12 06:55:52 on dist-test-slave-65mx"
server_key: "08716a575497f06922923357e9a535b0"
server_key_iv: "d356950f662fd4ad7b0b1ec809dc7572"
server_key_version: "encryptionkey@0"
I20260812 06:55:52.882995 30465 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260812 06:55:52.885743 30480 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 06:55:52.887198 30465 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260812 06:55:52.887375 30465 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786517687384214-19665-0/minicluster-data/ts-0/data,/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786517687384214-19665-0/minicluster-data/ts-0/wal
uuid: "34c798dc3ce04e9e93c80db51effaebc"
format_stamp: "Formatted at 2026-08-12 06:55:52 on dist-test-slave-65mx"
server_key: "08716a575497f06922923357e9a535b0"
server_key_iv: "d356950f662fd4ad7b0b1ec809dc7572"
server_key_version: "encryptionkey@0"
I20260812 06:55:52.887522 30465 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786517687384214-19665-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786517687384214-19665-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786517687384214-19665-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:55:52.905272 30465 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 06:55:52.906050 30465 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 06:55:52.906279 30465 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:55:52.906940 30465 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 06:55:52.908070 30465 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 06:55:52.908145 30465 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 06:55:52.908221 30465 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 06:55:52.908264 30465 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 06:55:52.918102 30465 rpc_server.cc:307] RPC server started. Bound to: 127.19.52.65:45415
I20260812 06:55:52.918135 30612 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.52.65:45415 every 8 connection(s)
I20260812 06:55:52.919040 30465 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786517687384214-19665-0/minicluster-data/ts-0/data/info.pb
I20260812 06:55:52.923050 19665 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu as pid 30465
I20260812 06:55:52.923192 19665 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786517687384214-19665-0/minicluster-data/ts-0/wal/instance
I20260812 06:55:52.923503 19665 external_mini_cluster.cc:1428] Setting key 225b407d7ebdda4308b8197dc38f1f9a
I20260812 06:55:52.925714 19665 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu
/tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786517687384214-19665-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786517687384214-19665-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786517687384214-19665-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786517687384214-19665-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.52.66:0
--local_ip_for_outbound_sockets=127.19.52.66
--webserver_interface=127.19.52.66
--webserver_port=0
--tserver_master_addrs=127.19.52.126:44069
--builtin_ntp_servers=127.19.52.84:44943
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260812 06:55:52.929432 30439 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:52.920932 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.52.65:54431 (local address 127.19.52.126:44069)
0812 06:55:52.921076 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0812 06:55:52.921081 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 06:55:52.922137 (+ 1056us) server_negotiation.cc:407] Connection header received
0812 06:55:52.923569 (+ 1432us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:55:52.923572 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:55:52.923645 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:55:52.923761 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:55:52.924837 (+ 1076us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:52.925655 (+ 818us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:52.926912 (+ 1257us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:52.927334 (+ 422us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:52.927456 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:55:52.928898 (+ 1442us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:55:52.928902 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:55:52.928905 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 06:55:52.928919 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:55:52.928946 (+ 27us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 06:55:52.928955 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:55:52.929038 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0812 06:55:52.929212 (+ 174us) server_negotiation.cc:299] Negotiation successful
0812 06:55:52.929284 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260812 06:55:52.930455 30615 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:52.921341 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.126:44069 (local address 127.19.52.65:54431)
0812 06:55:52.921984 (+ 643us) negotiation.cc:107] Waiting for socket to connect
0812 06:55:52.922025 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0812 06:55:52.923356 (+ 1331us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:55:52.923933 (+ 577us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:55:52.923940 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:55:52.924020 (+ 80us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:55:52.924657 (+ 637us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:55:52.924677 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:55:52.925835 (+ 1158us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:55:52.925840 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:55:52.926782 (+ 942us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:55:52.926792 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:55:52.927643 (+ 851us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:55:52.927648 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:55:52.927818 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:55:52.928527 (+ 709us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 06:55:52.928529 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:55:52.928632 (+ 103us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 06:55:52.928713 (+ 81us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 06:55:52.929040 (+ 327us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 06:55:52.929054 (+ 14us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 06:55:52.929060 (+ 6us) client_negotiation.cc:769] Sending connection context
0812 06:55:52.929211 (+ 151us) client_negotiation.cc:240] Negotiation successful
0812 06:55:52.929504 (+ 293us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":266,"thread_start_us":114,"threads_started":1}
I20260812 06:55:52.931759 30613 heartbeater.cc:347] Connected to a master server at 127.19.52.126:44069
I20260812 06:55:52.932019 30613 heartbeater.cc:464] Registering TS with master...
I20260812 06:55:52.932612 30613 heartbeater.cc:511] Master 127.19.52.126:44069 requested a full tablet report, sending...
I20260812 06:55:52.934372 30395 ts_manager.cc:194] Registered new tserver with Master: 34c798dc3ce04e9e93c80db51effaebc (127.19.52.65:45415)
I20260812 06:55:52.936213 30395 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.19.52.65:54431
W20260812 06:55:53.046988 30616 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:55:53.047302 30616 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:55:53.047343 30616 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 06:55:53.051518 30616 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 06:55:53.051600 30616 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 06:55:53.051659 30616 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.52.66
I20260812 06:55:53.056767 30616 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.52.84:44943
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786517687384214-19665-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786517687384214-19665-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.52.66:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786517687384214-19665-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.19.52.66
--webserver_port=0
--tserver_master_addrs=127.19.52.126:44069
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30616
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.52.66
--log_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786517687384214-19665-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 06:15:54 UTC on ccycloud.quasar-rnfzce.root.comops.site
build id 4581989
I20260812 06:55:53.057955 30616 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:55:53.059372 30616 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 06:55:53.070233 30622 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:55:53.070197 30621 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:55:53.070204 30625 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:55:53.070613 30616 server_base.cc:1034] running on GCE node
I20260812 06:55:53.071265 30616 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:55:53.071986 30616 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:55:53.073223 30616 hybrid_clock.cc:648] HybridClock initialized: now 1786517753073215 us; error 38 us; skew 500 ppm
I20260812 06:55:53.075203 30616 webserver.cc:460] Webserver started at http://127.19.52.66:35291/ using document root <none> and password file <none>
I20260812 06:55:53.075819 30616 fs_manager.cc:359] Metadata directory not provided
I20260812 06:55:53.075918 30616 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:55:53.076195 30616 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 06:55:53.078418 30616 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786517687384214-19665-0/minicluster-data/ts-1/data/instance:
uuid: "c6c13ae3b5754baf9a7c2c95fece6887"
format_stamp: "Formatted at 2026-08-12 06:55:53 on dist-test-slave-65mx"
server_key: "1686b5fe0972b4c8d9931deb3ed48cef"
server_key_iv: "f24ac672e71cd4a604374a8da40e1782"
server_key_version: "encryptionkey@0"
I20260812 06:55:53.079016 30616 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786517687384214-19665-0/minicluster-data/ts-1/wal/instance:
uuid: "c6c13ae3b5754baf9a7c2c95fece6887"
format_stamp: "Formatted at 2026-08-12 06:55:53 on dist-test-slave-65mx"
server_key: "1686b5fe0972b4c8d9931deb3ed48cef"
server_key_iv: "f24ac672e71cd4a604374a8da40e1782"
server_key_version: "encryptionkey@0"
I20260812 06:55:53.083042 30616 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.004s
I20260812 06:55:53.085865 30632 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 06:55:53.087159 30616 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260812 06:55:53.087309 30616 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786517687384214-19665-0/minicluster-data/ts-1/data,/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786517687384214-19665-0/minicluster-data/ts-1/wal
uuid: "c6c13ae3b5754baf9a7c2c95fece6887"
format_stamp: "Formatted at 2026-08-12 06:55:53 on dist-test-slave-65mx"
server_key: "1686b5fe0972b4c8d9931deb3ed48cef"
server_key_iv: "f24ac672e71cd4a604374a8da40e1782"
server_key_version: "encryptionkey@0"
I20260812 06:55:53.087440 30616 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786517687384214-19665-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786517687384214-19665-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786517687384214-19665-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:55:53.111491 30616 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 06:55:53.112300 30616 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 06:55:53.112535 30616 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:55:53.113261 30616 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 06:55:53.114251 30616 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 06:55:53.114326 30616 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 06:55:53.114404 30616 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 06:55:53.114449 30616 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 06:55:53.123824 30616 rpc_server.cc:307] RPC server started. Bound to: 127.19.52.66:46235
I20260812 06:55:53.123852 30752 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.52.66:46235 every 8 connection(s)
I20260812 06:55:53.124936 30616 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786517687384214-19665-0/minicluster-data/ts-1/data/info.pb
I20260812 06:55:53.134181 19665 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu as pid 30616
I20260812 06:55:53.134287 19665 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786517687384214-19665-0/minicluster-data/ts-1/wal/instance
I20260812 06:55:53.134675 19665 external_mini_cluster.cc:1428] Setting key 3cac9fd423589ee2f3b937c114fea6c5
I20260812 06:55:53.134830 30439 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:53.126997 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.52.66:39273 (local address 127.19.52.126:44069)
0812 06:55:53.127147 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0812 06:55:53.127151 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 06:55:53.128229 (+ 1078us) server_negotiation.cc:407] Connection header received
0812 06:55:53.129701 (+ 1472us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:55:53.129704 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:55:53.129754 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:55:53.129836 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:55:53.130784 (+ 948us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:53.131350 (+ 566us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:53.132609 (+ 1259us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:53.132849 (+ 240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:53.132940 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:55:53.134188 (+ 1248us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:55:53.134192 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:55:53.134194 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 06:55:53.134208 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:55:53.134228 (+ 20us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 06:55:53.134236 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:55:53.134335 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0812 06:55:53.134567 (+ 232us) server_negotiation.cc:299] Negotiation successful
0812 06:55:53.134659 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260812 06:55:53.135790 30755 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:53.127373 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.126:44069 (local address 127.19.52.66:39273)
0812 06:55:53.128043 (+ 670us) negotiation.cc:107] Waiting for socket to connect
0812 06:55:53.128086 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0812 06:55:53.129484 (+ 1398us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:55:53.130008 (+ 524us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:55:53.130015 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:55:53.130097 (+ 82us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:55:53.130597 (+ 500us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:55:53.130610 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:55:53.131495 (+ 885us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:55:53.131498 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:55:53.132437 (+ 939us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:55:53.132447 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:55:53.133000 (+ 553us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:55:53.133003 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:55:53.133157 (+ 154us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:55:53.133886 (+ 729us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 06:55:53.133888 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:55:53.133931 (+ 43us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 06:55:53.134000 (+ 69us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 06:55:53.134400 (+ 400us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 06:55:53.134409 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 06:55:53.134414 (+ 5us) client_negotiation.cc:769] Sending connection context
0812 06:55:53.134516 (+ 102us) client_negotiation.cc:240] Negotiation successful
0812 06:55:53.134802 (+ 286us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":284,"thread_start_us":110,"threads_started":1}
I20260812 06:55:53.137183 30753 heartbeater.cc:347] Connected to a master server at 127.19.52.126:44069
I20260812 06:55:53.137202 19665 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu
/tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786517687384214-19665-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786517687384214-19665-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786517687384214-19665-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786517687384214-19665-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.52.67:0
--local_ip_for_outbound_sockets=127.19.52.67
--webserver_interface=127.19.52.67
--webserver_port=0
--tserver_master_addrs=127.19.52.126:44069
--builtin_ntp_servers=127.19.52.84:44943
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260812 06:55:53.137449 30753 heartbeater.cc:464] Registering TS with master...
I20260812 06:55:53.138108 30753 heartbeater.cc:511] Master 127.19.52.126:44069 requested a full tablet report, sending...
I20260812 06:55:53.139317 30395 ts_manager.cc:194] Registered new tserver with Master: c6c13ae3b5754baf9a7c2c95fece6887 (127.19.52.66:46235)
I20260812 06:55:53.140177 30395 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.19.52.66:39273
W20260812 06:55:53.257987 30756 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:55:53.258316 30756 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:55:53.258358 30756 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 06:55:53.262799 30756 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 06:55:53.262938 30756 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 06:55:53.263049 30756 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.52.67
I20260812 06:55:53.269373 30756 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.52.84:44943
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786517687384214-19665-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786517687384214-19665-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.52.67:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786517687384214-19665-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.19.52.67
--webserver_port=0
--tserver_master_addrs=127.19.52.126:44069
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30756
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.52.67
--log_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786517687384214-19665-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 06:15:54 UTC on ccycloud.quasar-rnfzce.root.comops.site
build id 4581989
I20260812 06:55:53.270533 30756 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:55:53.271953 30756 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 06:55:53.282287 30762 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:55:53.282446 30761 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:55:53.282325 30765 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:55:53.282658 30756 server_base.cc:1034] running on GCE node
I20260812 06:55:53.283093 30756 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:55:53.283797 30756 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:55:53.284951 30756 hybrid_clock.cc:648] HybridClock initialized: now 1786517753284936 us; error 31 us; skew 500 ppm
I20260812 06:55:53.286846 30756 webserver.cc:460] Webserver started at http://127.19.52.67:41885/ using document root <none> and password file <none>
I20260812 06:55:53.287451 30756 fs_manager.cc:359] Metadata directory not provided
I20260812 06:55:53.287518 30756 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:55:53.287740 30756 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 06:55:53.289930 30756 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786517687384214-19665-0/minicluster-data/ts-2/data/instance:
uuid: "b76b7a2607a94215b0b50c89c79c4da3"
format_stamp: "Formatted at 2026-08-12 06:55:53 on dist-test-slave-65mx"
server_key: "89ab55d26ded735c43e41f9c98abfe54"
server_key_iv: "d202dd79c39ecd506376857f9e46db47"
server_key_version: "encryptionkey@0"
I20260812 06:55:53.290552 30756 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786517687384214-19665-0/minicluster-data/ts-2/wal/instance:
uuid: "b76b7a2607a94215b0b50c89c79c4da3"
format_stamp: "Formatted at 2026-08-12 06:55:53 on dist-test-slave-65mx"
server_key: "89ab55d26ded735c43e41f9c98abfe54"
server_key_iv: "d202dd79c39ecd506376857f9e46db47"
server_key_version: "encryptionkey@0"
I20260812 06:55:53.295007 30756 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260812 06:55:53.298342 30776 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 06:55:53.299793 30756 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260812 06:55:53.299950 30756 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786517687384214-19665-0/minicluster-data/ts-2/data,/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786517687384214-19665-0/minicluster-data/ts-2/wal
uuid: "b76b7a2607a94215b0b50c89c79c4da3"
format_stamp: "Formatted at 2026-08-12 06:55:53 on dist-test-slave-65mx"
server_key: "89ab55d26ded735c43e41f9c98abfe54"
server_key_iv: "d202dd79c39ecd506376857f9e46db47"
server_key_version: "encryptionkey@0"
I20260812 06:55:53.300096 30756 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786517687384214-19665-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786517687384214-19665-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786517687384214-19665-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:55:53.323283 30756 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 06:55:53.324209 30756 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 06:55:53.324450 30756 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:55:53.325161 30756 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 06:55:53.326282 30756 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 06:55:53.326370 30756 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 06:55:53.326463 30756 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 06:55:53.326511 30756 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 06:55:53.336436 30756 rpc_server.cc:307] RPC server started. Bound to: 127.19.52.67:41647
I20260812 06:55:53.336462 30896 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.52.67:41647 every 8 connection(s)
I20260812 06:55:53.337456 30756 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786517687384214-19665-0/minicluster-data/ts-2/data/info.pb
I20260812 06:55:53.346122 19665 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu as pid 30756
I20260812 06:55:53.346223 19665 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786517687384214-19665-0/minicluster-data/ts-2/wal/instance
I20260812 06:55:53.346562 19665 external_mini_cluster.cc:1428] Setting key a3817ff847c7597669ce35b6b281d47e
I20260812 06:55:53.346938 30439 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:53.339376 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.52.67:43413 (local address 127.19.52.126:44069)
0812 06:55:53.339511 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0812 06:55:53.339515 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 06:55:53.340503 (+ 988us) server_negotiation.cc:407] Connection header received
0812 06:55:53.341993 (+ 1490us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:55:53.341997 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:55:53.342060 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:55:53.342142 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:55:53.343042 (+ 900us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:53.343585 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:53.344781 (+ 1196us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:53.345005 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:53.345123 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:55:53.346411 (+ 1288us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:55:53.346414 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:55:53.346416 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 06:55:53.346429 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:55:53.346449 (+ 20us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 06:55:53.346457 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:55:53.346539 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0812 06:55:53.346718 (+ 179us) server_negotiation.cc:299] Negotiation successful
0812 06:55:53.346792 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260812 06:55:53.348045 30899 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:53.339716 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.126:44069 (local address 127.19.52.67:43413)
0812 06:55:53.340350 (+ 634us) negotiation.cc:107] Waiting for socket to connect
0812 06:55:53.340389 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0812 06:55:53.341768 (+ 1379us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:55:53.342313 (+ 545us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:55:53.342320 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:55:53.342400 (+ 80us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:55:53.342882 (+ 482us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:55:53.342894 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:55:53.343719 (+ 825us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:55:53.343722 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:55:53.344649 (+ 927us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:55:53.344658 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:55:53.345168 (+ 510us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:55:53.345172 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:55:53.345337 (+ 165us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:55:53.346093 (+ 756us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 06:55:53.346094 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:55:53.346135 (+ 41us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 06:55:53.346230 (+ 95us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 06:55:53.346541 (+ 311us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 06:55:53.346553 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 06:55:53.346562 (+ 9us) client_negotiation.cc:769] Sending connection context
0812 06:55:53.346716 (+ 154us) client_negotiation.cc:240] Negotiation successful
0812 06:55:53.347006 (+ 290us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":255,"thread_start_us":108,"threads_started":1}
I20260812 06:55:53.349308 30897 heartbeater.cc:347] Connected to a master server at 127.19.52.126:44069
I20260812 06:55:53.349617 30897 heartbeater.cc:464] Registering TS with master...
I20260812 06:55:53.350214 30897 heartbeater.cc:511] Master 127.19.52.126:44069 requested a full tablet report, sending...
I20260812 06:55:53.351301 30395 ts_manager.cc:194] Registered new tserver with Master: b76b7a2607a94215b0b50c89c79c4da3 (127.19.52.67:41647)
I20260812 06:55:53.351974 30395 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.19.52.67:43413
I20260812 06:55:53.361281 19665 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260812 06:55:53.365128 30439 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 06:55:53.363817 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36666 (local address 127.19.52.126:44069)
0812 06:55:53.363978 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0812 06:55:53.363983 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 06:55:53.364203 (+ 220us) server_negotiation.cc:407] Connection header received
0812 06:55:53.364422 (+ 219us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:55:53.364431 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:55:53.364481 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:55:53.364636 (+ 155us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:55:53.364918 (+ 282us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:36666: BlockingRecv error: recv got EOF from 127.0.0.1:36666 (error 108)
Metrics: {"server-negotiator.queue_time_us":64}
I20260812 06:55:53.365195 19665 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu with pid 30465
I20260812 06:55:53.379690 19665 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu with pid 30616
I20260812 06:55:53.391346 19665 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu with pid 30756
I20260812 06:55:53.403158 19665 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu with pid 30362
2026-08-12T06:55:53Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationInsecureCluster (996 ms)
[ RUN ] SecurityITest.TestRequireEncryptionInsecureCluster
2026-08-12T06:55:53Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T06:55:53Z Disabled control of system clock
I20260812 06:55:53.440263 19665 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu
/tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786517687384214-19665-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786517687384214-19665-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786517687384214-19665-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786517687384214-19665-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.52.126:45153
--webserver_interface=127.19.52.126
--webserver_port=0
--builtin_ntp_servers=127.19.52.84:35723
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.19.52.126:45153
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260812 06:55:53.558785 30910 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:55:53.559125 30910 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:55:53.559170 30910 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 06:55:53.563928 30910 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 06:55:53.564033 30910 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 06:55:53.564054 30910 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 06:55:53.564073 30910 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 06:55:53.564095 30910 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 06:55:53.569602 30910 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.52.84:35723
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786517687384214-19665-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786517687384214-19665-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.19.52.126:45153
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.52.126:45153
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786517687384214-19665-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.19.52.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30910
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786517687384214-19665-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 06:15:54 UTC on ccycloud.quasar-rnfzce.root.comops.site
build id 4581989
I20260812 06:55:53.570941 30910 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:55:53.572438 30910 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 06:55:53.581459 30919 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:55:53.581642 30910 server_base.cc:1034] running on GCE node
W20260812 06:55:53.581429 30918 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:55:53.581452 30921 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:55:53.582404 30910 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:55:53.583587 30910 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:55:53.584755 30910 hybrid_clock.cc:648] HybridClock initialized: now 1786517753584724 us; error 49 us; skew 500 ppm
I20260812 06:55:53.586925 30910 webserver.cc:460] Webserver started at http://127.19.52.126:43547/ using document root <none> and password file <none>
I20260812 06:55:53.587539 30910 fs_manager.cc:359] Metadata directory not provided
I20260812 06:55:53.587612 30910 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:55:53.587785 30910 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 06:55:53.589926 30910 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786517687384214-19665-0/minicluster-data/master-0/data/instance:
uuid: "88962b8cd0424e81bf1000d5696e45c8"
format_stamp: "Formatted at 2026-08-12 06:55:53 on dist-test-slave-65mx"
server_key: "f0477fb70e9e76fef32c31fee00d9cbb"
server_key_iv: "a2d1426662e1b5ce3674a5ea14902fd7"
server_key_version: "encryptionkey@0"
I20260812 06:55:53.590466 30910 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786517687384214-19665-0/minicluster-data/master-0/wal/instance:
uuid: "88962b8cd0424e81bf1000d5696e45c8"
format_stamp: "Formatted at 2026-08-12 06:55:53 on dist-test-slave-65mx"
server_key: "f0477fb70e9e76fef32c31fee00d9cbb"
server_key_iv: "a2d1426662e1b5ce3674a5ea14902fd7"
server_key_version: "encryptionkey@0"
I20260812 06:55:53.594396 30910 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260812 06:55:53.597460 30930 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 06:55:53.598773 30910 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 06:55:53.598924 30910 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786517687384214-19665-0/minicluster-data/master-0/data,/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786517687384214-19665-0/minicluster-data/master-0/wal
uuid: "88962b8cd0424e81bf1000d5696e45c8"
format_stamp: "Formatted at 2026-08-12 06:55:53 on dist-test-slave-65mx"
server_key: "f0477fb70e9e76fef32c31fee00d9cbb"
server_key_iv: "a2d1426662e1b5ce3674a5ea14902fd7"
server_key_version: "encryptionkey@0"
I20260812 06:55:53.599056 30910 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786517687384214-19665-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786517687384214-19665-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786517687384214-19665-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:55:53.604056 30910 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 06:55:53.604836 30910 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 06:55:53.605049 30910 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:55:53.612715 30910 rpc_server.cc:307] RPC server started. Bound to: 127.19.52.126:45153
I20260812 06:55:53.612735 30984 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.52.126:45153 every 8 connection(s)
I20260812 06:55:53.613727 30910 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786517687384214-19665-0/minicluster-data/master-0/data/info.pb
I20260812 06:55:53.617319 30985 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 06:55:53.617851 19665 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu as pid 30910
I20260812 06:55:53.617970 19665 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786517687384214-19665-0/minicluster-data/master-0/wal/instance
I20260812 06:55:53.618314 19665 external_mini_cluster.cc:1428] Setting key da6d559d24b45cd4d9061bd4ca27b691
I20260812 06:55:53.625321 30989 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:53.620047 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38366 (local address 127.19.52.126:45153)
0812 06:55:53.620940 (+ 893us) server_negotiation.cc:206] Beginning negotiation
0812 06:55:53.620973 (+ 33us) server_negotiation.cc:399] Waiting for connection header
0812 06:55:53.621060 (+ 87us) server_negotiation.cc:407] Connection header received
0812 06:55:53.621938 (+ 878us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:55:53.621963 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:55:53.622444 (+ 481us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:55:53.622828 (+ 384us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:55:53.623672 (+ 844us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:55:53.623678 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:55:53.623693 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 06:55:53.623705 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:55:53.623772 (+ 67us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 06:55:53.623795 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:55:53.623954 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0812 06:55:53.624064 (+ 110us) server_negotiation.cc:299] Negotiation successful
0812 06:55:53.624322 (+ 258us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":323,"thread_start_us":138,"threads_started":1}
I20260812 06:55:53.625340 30985 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 88962b8cd0424e81bf1000d5696e45c8: Bootstrap starting.
I20260812 06:55:53.628017 30985 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 88962b8cd0424e81bf1000d5696e45c8: Neither blocks nor log segments found. Creating new log.
I20260812 06:55:53.628890 30985 log.cc:824] T 00000000000000000000000000000000 P 88962b8cd0424e81bf1000d5696e45c8: Log is configured to *not* fsync() on all Append() calls
I20260812 06:55:53.630983 30985 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 88962b8cd0424e81bf1000d5696e45c8: No bootstrap required, opened a new log
I20260812 06:55:53.635134 30985 raft_consensus.cc:348] T 00000000000000000000000000000000 P 88962b8cd0424e81bf1000d5696e45c8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "88962b8cd0424e81bf1000d5696e45c8" member_type: VOTER last_known_addr { host: "127.19.52.126" port: 45153 } }
I20260812 06:55:53.635391 30985 raft_consensus.cc:374] T 00000000000000000000000000000000 P 88962b8cd0424e81bf1000d5696e45c8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 06:55:53.635427 30985 raft_consensus.cc:729] T 00000000000000000000000000000000 P 88962b8cd0424e81bf1000d5696e45c8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 88962b8cd0424e81bf1000d5696e45c8, State: Initialized, Role: FOLLOWER
I20260812 06:55:53.635870 30985 consensus_queue.cc:260] T 00000000000000000000000000000000 P 88962b8cd0424e81bf1000d5696e45c8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "88962b8cd0424e81bf1000d5696e45c8" member_type: VOTER last_known_addr { host: "127.19.52.126" port: 45153 } }
I20260812 06:55:53.636008 30985 raft_consensus.cc:388] T 00000000000000000000000000000000 P 88962b8cd0424e81bf1000d5696e45c8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 06:55:53.636051 30985 raft_consensus.cc:482] T 00000000000000000000000000000000 P 88962b8cd0424e81bf1000d5696e45c8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 06:55:53.636119 30985 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 88962b8cd0424e81bf1000d5696e45c8 [term 0 FOLLOWER]: Advancing to term 1
I20260812 06:55:53.637213 30985 raft_consensus.cc:504] T 00000000000000000000000000000000 P 88962b8cd0424e81bf1000d5696e45c8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "88962b8cd0424e81bf1000d5696e45c8" member_type: VOTER last_known_addr { host: "127.19.52.126" port: 45153 } }
I20260812 06:55:53.637547 30985 leader_election.cc:302] T 00000000000000000000000000000000 P 88962b8cd0424e81bf1000d5696e45c8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 88962b8cd0424e81bf1000d5696e45c8; no voters:
I20260812 06:55:53.637830 30985 leader_election.cc:288] T 00000000000000000000000000000000 P 88962b8cd0424e81bf1000d5696e45c8 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 06:55:53.637995 30991 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 88962b8cd0424e81bf1000d5696e45c8 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 06:55:53.638286 30991 raft_consensus.cc:686] T 00000000000000000000000000000000 P 88962b8cd0424e81bf1000d5696e45c8 [term 1 LEADER]: Becoming Leader. State: Replica: 88962b8cd0424e81bf1000d5696e45c8, State: Running, Role: LEADER
I20260812 06:55:53.638792 30991 consensus_queue.cc:237] T 00000000000000000000000000000000 P 88962b8cd0424e81bf1000d5696e45c8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "88962b8cd0424e81bf1000d5696e45c8" member_type: VOTER last_known_addr { host: "127.19.52.126" port: 45153 } }
I20260812 06:55:53.639429 30985 sys_catalog.cc:564] T 00000000000000000000000000000000 P 88962b8cd0424e81bf1000d5696e45c8 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 06:55:53.640779 30992 sys_catalog.cc:455] T 00000000000000000000000000000000 P 88962b8cd0424e81bf1000d5696e45c8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "88962b8cd0424e81bf1000d5696e45c8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "88962b8cd0424e81bf1000d5696e45c8" member_type: VOTER last_known_addr { host: "127.19.52.126" port: 45153 } } }
I20260812 06:55:53.640916 30992 sys_catalog.cc:458] T 00000000000000000000000000000000 P 88962b8cd0424e81bf1000d5696e45c8 [sys.catalog]: This master's current role is: LEADER
I20260812 06:55:53.641372 31002 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 06:55:53.641330 30994 sys_catalog.cc:455] T 00000000000000000000000000000000 P 88962b8cd0424e81bf1000d5696e45c8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 88962b8cd0424e81bf1000d5696e45c8. Latest consensus state: current_term: 1 leader_uuid: "88962b8cd0424e81bf1000d5696e45c8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "88962b8cd0424e81bf1000d5696e45c8" member_type: VOTER last_known_addr { host: "127.19.52.126" port: 45153 } } }
I20260812 06:55:53.641530 30994 sys_catalog.cc:458] T 00000000000000000000000000000000 P 88962b8cd0424e81bf1000d5696e45c8 [sys.catalog]: This master's current role is: LEADER
I20260812 06:55:53.644323 31002 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 06:55:53.650722 31002 catalog_manager.cc:1347] Generated new cluster ID: 85cb826c9a1b447b9dda7d72cd17392f
I20260812 06:55:53.650825 31002 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 06:55:53.670429 31002 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 06:55:53.670799 31002 catalog_manager.cc:1504] Loading token signing keys...
I20260812 06:55:53.675806 31002 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 88962b8cd0424e81bf1000d5696e45c8: Generated new TSK 0
I20260812 06:55:53.676600 31002 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 06:55:53.683308 19665 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu
/tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786517687384214-19665-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786517687384214-19665-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786517687384214-19665-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786517687384214-19665-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.52.65:0
--local_ip_for_outbound_sockets=127.19.52.65
--webserver_interface=127.19.52.65
--webserver_port=0
--tserver_master_addrs=127.19.52.126:45153
--builtin_ntp_servers=127.19.52.84:35723
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260812 06:55:53.813299 31012 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:55:53.813586 31012 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:55:53.813637 31012 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 06:55:53.817785 31012 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 06:55:53.817850 31012 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 06:55:53.817909 31012 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.52.65
I20260812 06:55:53.822817 31012 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.52.84:35723
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786517687384214-19665-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786517687384214-19665-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.52.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786517687384214-19665-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.19.52.65
--webserver_port=0
--tserver_master_addrs=127.19.52.126:45153
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31012
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.52.65
--log_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786517687384214-19665-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 06:15:54 UTC on ccycloud.quasar-rnfzce.root.comops.site
build id 4581989
I20260812 06:55:53.823920 31012 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:55:53.825426 31012 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 06:55:53.835896 31022 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:55:53.835898 31018 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:55:53.836048 31012 server_base.cc:1034] running on GCE node
W20260812 06:55:53.835896 31019 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:55:53.836714 31012 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:55:53.837456 31012 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:55:53.838649 31012 hybrid_clock.cc:648] HybridClock initialized: now 1786517753838635 us; error 31 us; skew 500 ppm
I20260812 06:55:53.840876 31012 webserver.cc:460] Webserver started at http://127.19.52.65:38227/ using document root <none> and password file <none>
I20260812 06:55:53.841600 31012 fs_manager.cc:359] Metadata directory not provided
I20260812 06:55:53.841691 31012 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:55:53.841914 31012 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 06:55:53.844092 31012 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786517687384214-19665-0/minicluster-data/ts-0/data/instance:
uuid: "00f8730066cf46e5a2c3bb87f524c3e4"
format_stamp: "Formatted at 2026-08-12 06:55:53 on dist-test-slave-65mx"
server_key: "59be159aa164922204d0a72b9ba3733e"
server_key_iv: "cdfc02c9e8dc9f9452ce658dabdd4235"
server_key_version: "encryptionkey@0"
I20260812 06:55:53.844765 31012 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786517687384214-19665-0/minicluster-data/ts-0/wal/instance:
uuid: "00f8730066cf46e5a2c3bb87f524c3e4"
format_stamp: "Formatted at 2026-08-12 06:55:53 on dist-test-slave-65mx"
server_key: "59be159aa164922204d0a72b9ba3733e"
server_key_iv: "cdfc02c9e8dc9f9452ce658dabdd4235"
server_key_version: "encryptionkey@0"
I20260812 06:55:53.848939 31012 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260812 06:55:53.852003 31028 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 06:55:53.853406 31012 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 06:55:53.853521 31012 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786517687384214-19665-0/minicluster-data/ts-0/data,/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786517687384214-19665-0/minicluster-data/ts-0/wal
uuid: "00f8730066cf46e5a2c3bb87f524c3e4"
format_stamp: "Formatted at 2026-08-12 06:55:53 on dist-test-slave-65mx"
server_key: "59be159aa164922204d0a72b9ba3733e"
server_key_iv: "cdfc02c9e8dc9f9452ce658dabdd4235"
server_key_version: "encryptionkey@0"
I20260812 06:55:53.853783 31012 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786517687384214-19665-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786517687384214-19665-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786517687384214-19665-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:55:53.859575 31012 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 06:55:53.860296 31012 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 06:55:53.860522 31012 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:55:53.861205 31012 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 06:55:53.862285 31012 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 06:55:53.862339 31012 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 06:55:53.862383 31012 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 06:55:53.862447 31012 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 06:55:53.872923 31012 rpc_server.cc:307] RPC server started. Bound to: 127.19.52.65:41631
I20260812 06:55:53.873119 31153 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.52.65:41631 every 8 connection(s)
I20260812 06:55:53.873996 31012 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786517687384214-19665-0/minicluster-data/ts-0/data/info.pb
I20260812 06:55:53.880437 30989 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:53.876012 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.52.65:42369 (local address 127.19.52.126:45153)
0812 06:55:53.876173 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0812 06:55:53.876177 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 06:55:53.877311 (+ 1134us) server_negotiation.cc:407] Connection header received
0812 06:55:53.878643 (+ 1332us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:55:53.878647 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:55:53.878696 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:55:53.878786 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:55:53.879936 (+ 1150us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:55:53.879939 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:55:53.879942 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 06:55:53.879955 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:55:53.879975 (+ 20us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 06:55:53.879984 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:55:53.880092 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0812 06:55:53.880254 (+ 162us) server_negotiation.cc:299] Negotiation successful
0812 06:55:53.880302 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260812 06:55:53.881309 31156 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:53.876372 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.126:45153 (local address 127.19.52.65:42369)
0812 06:55:53.877138 (+ 766us) negotiation.cc:107] Waiting for socket to connect
0812 06:55:53.877198 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0812 06:55:53.878424 (+ 1226us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:55:53.878964 (+ 540us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:55:53.878970 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:55:53.879055 (+ 85us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:55:53.879677 (+ 622us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 06:55:53.879678 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:55:53.879719 (+ 41us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 06:55:53.879786 (+ 67us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 06:55:53.880084 (+ 298us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 06:55:53.880101 (+ 17us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 06:55:53.880110 (+ 9us) client_negotiation.cc:769] Sending connection context
0812 06:55:53.880288 (+ 178us) client_negotiation.cc:240] Negotiation successful
0812 06:55:53.880525 (+ 237us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":345,"thread_start_us":192,"threads_started":1}
I20260812 06:55:53.882433 19665 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu as pid 31012
I20260812 06:55:53.882561 19665 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786517687384214-19665-0/minicluster-data/ts-0/wal/instance
I20260812 06:55:53.882555 31154 heartbeater.cc:347] Connected to a master server at 127.19.52.126:45153
I20260812 06:55:53.882797 31154 heartbeater.cc:464] Registering TS with master...
I20260812 06:55:53.882908 19665 external_mini_cluster.cc:1428] Setting key 73943fb08b4eb8082efa8d01b1895914
I20260812 06:55:53.883279 31154 heartbeater.cc:511] Master 127.19.52.126:45153 requested a full tablet report, sending...
I20260812 06:55:53.885032 30948 ts_manager.cc:194] Registered new tserver with Master: 00f8730066cf46e5a2c3bb87f524c3e4 (127.19.52.65:41631)
I20260812 06:55:53.885085 19665 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu
/tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786517687384214-19665-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786517687384214-19665-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786517687384214-19665-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786517687384214-19665-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.52.66:0
--local_ip_for_outbound_sockets=127.19.52.66
--webserver_interface=127.19.52.66
--webserver_port=0
--tserver_master_addrs=127.19.52.126:45153
--builtin_ntp_servers=127.19.52.84:35723
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260812 06:55:54.001586 31158 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:55:54.001885 31158 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:55:54.001961 31158 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 06:55:54.006268 31158 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 06:55:54.006357 31158 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 06:55:54.006453 31158 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.52.66
I20260812 06:55:54.011301 31158 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.52.84:35723
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786517687384214-19665-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786517687384214-19665-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.52.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786517687384214-19665-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.19.52.66
--webserver_port=0
--tserver_master_addrs=127.19.52.126:45153
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31158
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.52.66
--log_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786517687384214-19665-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 06:15:54 UTC on ccycloud.quasar-rnfzce.root.comops.site
build id 4581989
I20260812 06:55:54.012462 31158 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:55:54.013890 31158 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 06:55:54.023578 31167 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:55:54.023640 31164 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:55:54.023586 31165 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:55:54.023867 31158 server_base.cc:1034] running on GCE node
I20260812 06:55:54.024322 31158 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:55:54.025068 31158 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:55:54.026271 31158 hybrid_clock.cc:648] HybridClock initialized: now 1786517754026259 us; error 31 us; skew 500 ppm
I20260812 06:55:54.028163 31158 webserver.cc:460] Webserver started at http://127.19.52.66:35349/ using document root <none> and password file <none>
I20260812 06:55:54.028826 31158 fs_manager.cc:359] Metadata directory not provided
I20260812 06:55:54.028923 31158 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:55:54.029191 31158 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 06:55:54.031383 31158 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786517687384214-19665-0/minicluster-data/ts-1/data/instance:
uuid: "91de7cc493034bfb91a031d164142b2e"
format_stamp: "Formatted at 2026-08-12 06:55:54 on dist-test-slave-65mx"
server_key: "c865ccdb062a4f39e13917ad78bd94df"
server_key_iv: "c360fc75b2d82de8fa2c44e51cd54beb"
server_key_version: "encryptionkey@0"
I20260812 06:55:54.031960 31158 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786517687384214-19665-0/minicluster-data/ts-1/wal/instance:
uuid: "91de7cc493034bfb91a031d164142b2e"
format_stamp: "Formatted at 2026-08-12 06:55:54 on dist-test-slave-65mx"
server_key: "c865ccdb062a4f39e13917ad78bd94df"
server_key_iv: "c360fc75b2d82de8fa2c44e51cd54beb"
server_key_version: "encryptionkey@0"
I20260812 06:55:54.036051 31158 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260812 06:55:54.039160 31173 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 06:55:54.040820 31158 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 06:55:54.040977 31158 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786517687384214-19665-0/minicluster-data/ts-1/data,/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786517687384214-19665-0/minicluster-data/ts-1/wal
uuid: "91de7cc493034bfb91a031d164142b2e"
format_stamp: "Formatted at 2026-08-12 06:55:54 on dist-test-slave-65mx"
server_key: "c865ccdb062a4f39e13917ad78bd94df"
server_key_iv: "c360fc75b2d82de8fa2c44e51cd54beb"
server_key_version: "encryptionkey@0"
I20260812 06:55:54.041177 31158 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786517687384214-19665-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786517687384214-19665-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786517687384214-19665-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:55:54.046437 31158 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 06:55:54.047168 31158 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 06:55:54.047480 31158 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:55:54.048105 31158 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 06:55:54.049321 31158 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 06:55:54.049402 31158 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 06:55:54.049476 31158 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 06:55:54.049527 31158 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 06:55:54.060658 31158 rpc_server.cc:307] RPC server started. Bound to: 127.19.52.66:41309
I20260812 06:55:54.060680 31299 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.52.66:41309 every 8 connection(s)
I20260812 06:55:54.061748 31158 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786517687384214-19665-0/minicluster-data/ts-1/data/info.pb
I20260812 06:55:54.063030 19665 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu as pid 31158
I20260812 06:55:54.063169 19665 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786517687384214-19665-0/minicluster-data/ts-1/wal/instance
I20260812 06:55:54.063493 19665 external_mini_cluster.cc:1428] Setting key e24fe6f12c006513cb133d875297bef5
I20260812 06:55:54.065725 19665 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu
/tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786517687384214-19665-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786517687384214-19665-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786517687384214-19665-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786517687384214-19665-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.52.67:0
--local_ip_for_outbound_sockets=127.19.52.67
--webserver_interface=127.19.52.67
--webserver_port=0
--tserver_master_addrs=127.19.52.126:45153
--builtin_ntp_servers=127.19.52.84:35723
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
I20260812 06:55:54.069258 30989 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:54.065764 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.52.66:60559 (local address 127.19.52.126:45153)
0812 06:55:54.065900 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0812 06:55:54.065904 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 06:55:54.065921 (+ 17us) server_negotiation.cc:407] Connection header received
0812 06:55:54.067186 (+ 1265us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:55:54.067189 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:55:54.067238 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:55:54.067349 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:55:54.068662 (+ 1313us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:55:54.068666 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:55:54.068669 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 06:55:54.068689 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:55:54.068711 (+ 22us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 06:55:54.068720 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:55:54.068825 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0812 06:55:54.069091 (+ 266us) server_negotiation.cc:299] Negotiation successful
0812 06:55:54.069143 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260812 06:55:54.070508 31304 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:54.064645 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.126:45153 (local address 127.19.52.66:60559)
0812 06:55:54.065545 (+ 900us) negotiation.cc:107] Waiting for socket to connect
0812 06:55:54.065615 (+ 70us) client_negotiation.cc:174] Beginning negotiation
0812 06:55:54.066933 (+ 1318us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:55:54.067537 (+ 604us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:55:54.067545 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:55:54.067643 (+ 98us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:55:54.068316 (+ 673us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 06:55:54.068318 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:55:54.068364 (+ 46us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 06:55:54.068456 (+ 92us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 06:55:54.068827 (+ 371us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 06:55:54.068841 (+ 14us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 06:55:54.068855 (+ 14us) client_negotiation.cc:769] Sending connection context
0812 06:55:54.069111 (+ 256us) client_negotiation.cc:240] Negotiation successful
0812 06:55:54.069384 (+ 273us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":316,"thread_start_us":141,"threads_started":1}
I20260812 06:55:54.072171 31300 heartbeater.cc:347] Connected to a master server at 127.19.52.126:45153
I20260812 06:55:54.072456 31300 heartbeater.cc:464] Registering TS with master...
I20260812 06:55:54.072985 31300 heartbeater.cc:511] Master 127.19.52.126:45153 requested a full tablet report, sending...
I20260812 06:55:54.074321 30948 ts_manager.cc:194] Registered new tserver with Master: 91de7cc493034bfb91a031d164142b2e (127.19.52.66:41309)
W20260812 06:55:54.185945 31305 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:55:54.186262 31305 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:55:54.186337 31305 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 06:55:54.190584 31305 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 06:55:54.190667 31305 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 06:55:54.190759 31305 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.52.67
I20260812 06:55:54.195566 31305 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.52.84:35723
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786517687384214-19665-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786517687384214-19665-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.52.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786517687384214-19665-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.19.52.67
--webserver_port=0
--tserver_master_addrs=127.19.52.126:45153
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31305
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.52.67
--log_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786517687384214-19665-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 06:15:54 UTC on ccycloud.quasar-rnfzce.root.comops.site
build id 4581989
I20260812 06:55:54.196821 31305 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:55:54.198408 31305 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 06:55:54.209141 31311 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:55:54.209180 31314 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:55:54.209156 31312 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:55:54.209936 31305 server_base.cc:1034] running on GCE node
I20260812 06:55:54.210386 31305 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:55:54.211130 31305 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:55:54.212352 31305 hybrid_clock.cc:648] HybridClock initialized: now 1786517754212321 us; error 52 us; skew 500 ppm
I20260812 06:55:54.214556 31305 webserver.cc:460] Webserver started at http://127.19.52.67:42111/ using document root <none> and password file <none>
I20260812 06:55:54.215222 31305 fs_manager.cc:359] Metadata directory not provided
I20260812 06:55:54.215328 31305 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:55:54.215551 31305 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 06:55:54.217852 31305 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786517687384214-19665-0/minicluster-data/ts-2/data/instance:
uuid: "91cd4f299aeb4889a1c89a25621dfdac"
format_stamp: "Formatted at 2026-08-12 06:55:54 on dist-test-slave-65mx"
server_key: "6322d1fed2ff430411fbbbbe93feb18e"
server_key_iv: "e40dbb4ebed2243209f63ac43bb673a5"
server_key_version: "encryptionkey@0"
I20260812 06:55:54.218452 31305 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786517687384214-19665-0/minicluster-data/ts-2/wal/instance:
uuid: "91cd4f299aeb4889a1c89a25621dfdac"
format_stamp: "Formatted at 2026-08-12 06:55:54 on dist-test-slave-65mx"
server_key: "6322d1fed2ff430411fbbbbe93feb18e"
server_key_iv: "e40dbb4ebed2243209f63ac43bb673a5"
server_key_version: "encryptionkey@0"
I20260812 06:55:54.222644 31305 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260812 06:55:54.225600 31320 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 06:55:54.227072 31305 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 06:55:54.227239 31305 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786517687384214-19665-0/minicluster-data/ts-2/data,/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786517687384214-19665-0/minicluster-data/ts-2/wal
uuid: "91cd4f299aeb4889a1c89a25621dfdac"
format_stamp: "Formatted at 2026-08-12 06:55:54 on dist-test-slave-65mx"
server_key: "6322d1fed2ff430411fbbbbe93feb18e"
server_key_iv: "e40dbb4ebed2243209f63ac43bb673a5"
server_key_version: "encryptionkey@0"
I20260812 06:55:54.227380 31305 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786517687384214-19665-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786517687384214-19665-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786517687384214-19665-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:55:54.232299 31305 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 06:55:54.233009 31305 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 06:55:54.233193 31305 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:55:54.233760 31305 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 06:55:54.234688 31305 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 06:55:54.234760 31305 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 06:55:54.234830 31305 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 06:55:54.234882 31305 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 06:55:54.244750 31305 rpc_server.cc:307] RPC server started. Bound to: 127.19.52.67:44707
I20260812 06:55:54.244781 31442 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.52.67:44707 every 8 connection(s)
I20260812 06:55:54.245838 31305 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786517687384214-19665-0/minicluster-data/ts-2/data/info.pb
I20260812 06:55:54.252722 30989 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:54.247930 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.52.67:40777 (local address 127.19.52.126:45153)
0812 06:55:54.248099 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0812 06:55:54.248103 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 06:55:54.249356 (+ 1253us) server_negotiation.cc:407] Connection header received
0812 06:55:54.250741 (+ 1385us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:55:54.250745 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:55:54.250795 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:55:54.250899 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:55:54.252106 (+ 1207us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:55:54.252109 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:55:54.252112 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 06:55:54.252126 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:55:54.252153 (+ 27us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 06:55:54.252168 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:55:54.252275 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0812 06:55:54.252412 (+ 137us) server_negotiation.cc:299] Negotiation successful
0812 06:55:54.252575 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260812 06:55:54.253708 31445 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:54.248350 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.126:45153 (local address 127.19.52.67:40777)
0812 06:55:54.249188 (+ 838us) negotiation.cc:107] Waiting for socket to connect
0812 06:55:54.249232 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0812 06:55:54.250506 (+ 1274us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:55:54.251078 (+ 572us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:55:54.251084 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:55:54.251173 (+ 89us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:55:54.251819 (+ 646us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 06:55:54.251821 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:55:54.251864 (+ 43us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 06:55:54.251945 (+ 81us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 06:55:54.252255 (+ 310us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 06:55:54.252264 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 06:55:54.252270 (+ 6us) client_negotiation.cc:769] Sending connection context
0812 06:55:54.252537 (+ 267us) client_negotiation.cc:240] Negotiation successful
0812 06:55:54.252825 (+ 288us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":382,"thread_start_us":189,"threads_started":1}
I20260812 06:55:54.254741 19665 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu as pid 31305
I20260812 06:55:54.254841 19665 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786517687384214-19665-0/minicluster-data/ts-2/wal/instance
I20260812 06:55:54.255149 31443 heartbeater.cc:347] Connected to a master server at 127.19.52.126:45153
I20260812 06:55:54.255208 19665 external_mini_cluster.cc:1428] Setting key 4908fbd4f8d5692e3bd19194b9d49ba4
I20260812 06:55:54.255414 31443 heartbeater.cc:464] Registering TS with master...
I20260812 06:55:54.255978 31443 heartbeater.cc:511] Master 127.19.52.126:45153 requested a full tablet report, sending...
I20260812 06:55:54.257274 30947 ts_manager.cc:194] Registered new tserver with Master: 91cd4f299aeb4889a1c89a25621dfdac (127.19.52.67:44707)
I20260812 06:55:54.270143 19665 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260812 06:55:54.273599 30989 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 06:55:54.272400 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38374 (local address 127.19.52.126:45153)
0812 06:55:54.272642 (+ 242us) server_negotiation.cc:206] Beginning negotiation
0812 06:55:54.272647 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 06:55:54.272811 (+ 164us) server_negotiation.cc:407] Connection header received
0812 06:55:54.272974 (+ 163us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:55:54.272978 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:55:54.273025 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:55:54.273168 (+ 143us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:55:54.273424 (+ 256us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:38374: BlockingRecv error: recv got EOF from 127.0.0.1:38374 (error 108)
Metrics: {"server-negotiator.queue_time_us":115}
I20260812 06:55:54.274145 19665 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu with pid 31012
I20260812 06:55:54.284334 19665 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu with pid 31158
I20260812 06:55:54.291085 19665 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu with pid 31305
I20260812 06:55:54.301986 19665 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu with pid 30910
2026-08-12T06:55:54Z chronyd exiting
[ OK ] SecurityITest.TestRequireEncryptionInsecureCluster (897 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationSecureCluster
Loading random data
Initializing database '/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786517687384214-19665-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 06:55:54 dist-test-slave-65mx krb5kdc[31453](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 06:55:54 dist-test-slave-65mx krb5kdc[31453](info): set up 2 sockets
Aug 12 06:55:54 dist-test-slave-65mx krb5kdc[31453](info): commencing operation
krb5kdc: starting...
W20260812 06:55:55.509008 19665 mini_kdc.cc:121] Time spent starting KDC: real 1.182s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786517687384214-19665-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786517687384214-19665-0/krb5kdc/test-user.keytab.
Aug 12 06:55:55 dist-test-slave-65mx krb5kdc[31453](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517755, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T06:55:55Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T06:55:55Z Disabled control of system clock
WARNING: no policy specified for kudu/127.19.52.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.52.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.52.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786517687384214-19665-0/krb5kdc/kudu_127.19.52.126.keytab.
Entry for principal kudu/127.19.52.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786517687384214-19665-0/krb5kdc/kudu_127.19.52.126.keytab.
I20260812 06:55:55.653394 19665 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu
/tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786517687384214-19665-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786517687384214-19665-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786517687384214-19665-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786517687384214-19665-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.52.126:42435
--webserver_interface=127.19.52.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786517687384214-19665-0/krb5kdc/kudu_127.19.52.126.keytab
--principal=kudu/127.19.52.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.52.84:40101
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.19.52.126:42435
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786517687384214-19665-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786517687384214-19665-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786517687384214-19665-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 06:55:55.771153 31466 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:55:55.771461 31466 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:55:55.771505 31466 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 06:55:55.775784 31466 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 06:55:55.775869 31466 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 06:55:55.775889 31466 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 06:55:55.775907 31466 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 06:55:55.775923 31466 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 06:55:55.781761 31466 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.52.84:40101
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786517687384214-19665-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786517687384214-19665-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.19.52.126:42435
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786517687384214-19665-0/krb5kdc/kudu_127.19.52.126.keytab
--principal=kudu/127.19.52.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.52.126:42435
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786517687384214-19665-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.19.52.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31466
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786517687384214-19665-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 06:15:54 UTC on ccycloud.quasar-rnfzce.root.comops.site
build id 4581989
I20260812 06:55:55.782899 31466 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:55:55.783953 31466 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260812 06:55:55.792603 31466 server_base.cc:1034] running on GCE node
W20260812 06:55:55.792572 31471 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:55:55.792589 31472 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:55:55.792644 31475 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:55:55.793346 31466 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:55:55.794486 31466 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:55:55.795694 31466 hybrid_clock.cc:648] HybridClock initialized: now 1786517755795664 us; error 41 us; skew 500 ppm
Aug 12 06:55:55 dist-test-slave-65mx krb5kdc[31453](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517755, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.52.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 06:55:55.803901 31466 init.cc:377] Logged in from keytab as kudu/127.19.52.126@KRBTEST.COM (short username kudu)
I20260812 06:55:55.805060 31466 webserver.cc:460] Webserver started at http://127.19.52.126:45629/ using document root <none> and password file <none>
I20260812 06:55:55.805675 31466 fs_manager.cc:359] Metadata directory not provided
I20260812 06:55:55.805774 31466 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:55:55.806020 31466 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 06:55:55.808193 31466 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786517687384214-19665-0/minicluster-data/master-0/data/instance:
uuid: "dffab2aaa41a4948ba46209085f4ca00"
format_stamp: "Formatted at 2026-08-12 06:55:55 on dist-test-slave-65mx"
server_key: "73d4a84678486a08f17406085260d222"
server_key_iv: "ea974d20fe8a052ae25bf038ee93903f"
server_key_version: "encryptionkey@0"
I20260812 06:55:55.808848 31466 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786517687384214-19665-0/minicluster-data/master-0/wal/instance:
uuid: "dffab2aaa41a4948ba46209085f4ca00"
format_stamp: "Formatted at 2026-08-12 06:55:55 on dist-test-slave-65mx"
server_key: "73d4a84678486a08f17406085260d222"
server_key_iv: "ea974d20fe8a052ae25bf038ee93903f"
server_key_version: "encryptionkey@0"
I20260812 06:55:55.812918 31466 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.002s
I20260812 06:55:55.815515 31484 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 06:55:55.816969 31466 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 06:55:55.817119 31466 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786517687384214-19665-0/minicluster-data/master-0/data,/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786517687384214-19665-0/minicluster-data/master-0/wal
uuid: "dffab2aaa41a4948ba46209085f4ca00"
format_stamp: "Formatted at 2026-08-12 06:55:55 on dist-test-slave-65mx"
server_key: "73d4a84678486a08f17406085260d222"
server_key_iv: "ea974d20fe8a052ae25bf038ee93903f"
server_key_version: "encryptionkey@0"
I20260812 06:55:55.817245 31466 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786517687384214-19665-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786517687384214-19665-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786517687384214-19665-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:55:55.829885 31466 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 06:55:55.833403 31466 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 06:55:55.833643 31466 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:55:55.841532 31466 rpc_server.cc:307] RPC server started. Bound to: 127.19.52.126:42435
I20260812 06:55:55.841852 31545 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.52.126:42435 every 8 connection(s)
I20260812 06:55:55.842670 31466 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786517687384214-19665-0/minicluster-data/master-0/data/info.pb
I20260812 06:55:55.845554 31546 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 06:55:55.851296 31546 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dffab2aaa41a4948ba46209085f4ca00: Bootstrap starting.
I20260812 06:55:55.852142 19665 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu as pid 31466
I20260812 06:55:55.852285 19665 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786517687384214-19665-0/minicluster-data/master-0/wal/instance
I20260812 06:55:55.852644 19665 external_mini_cluster.cc:1428] Setting key 59fe826c52624022db5e2c22784af808
I20260812 06:55:55.854563 31546 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P dffab2aaa41a4948ba46209085f4ca00: Neither blocks nor log segments found. Creating new log.
I20260812 06:55:55.855351 31546 log.cc:824] T 00000000000000000000000000000000 P dffab2aaa41a4948ba46209085f4ca00: Log is configured to *not* fsync() on all Append() calls
I20260812 06:55:55.857975 31546 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dffab2aaa41a4948ba46209085f4ca00: No bootstrap required, opened a new log
Aug 12 06:55:55 dist-test-slave-65mx krb5kdc[31453](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517755, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.19.52.126@KRBTEST.COM
I20260812 06:55:55.863047 31546 raft_consensus.cc:348] T 00000000000000000000000000000000 P dffab2aaa41a4948ba46209085f4ca00 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dffab2aaa41a4948ba46209085f4ca00" member_type: VOTER last_known_addr { host: "127.19.52.126" port: 42435 } }
I20260812 06:55:55.863411 31546 raft_consensus.cc:374] T 00000000000000000000000000000000 P dffab2aaa41a4948ba46209085f4ca00 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 06:55:55.863495 31546 raft_consensus.cc:729] T 00000000000000000000000000000000 P dffab2aaa41a4948ba46209085f4ca00 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dffab2aaa41a4948ba46209085f4ca00, State: Initialized, Role: FOLLOWER
I20260812 06:55:55.863984 31546 consensus_queue.cc:260] T 00000000000000000000000000000000 P dffab2aaa41a4948ba46209085f4ca00 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dffab2aaa41a4948ba46209085f4ca00" member_type: VOTER last_known_addr { host: "127.19.52.126" port: 42435 } }
I20260812 06:55:55.864144 31546 raft_consensus.cc:388] T 00000000000000000000000000000000 P dffab2aaa41a4948ba46209085f4ca00 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 06:55:55.864236 31546 raft_consensus.cc:482] T 00000000000000000000000000000000 P dffab2aaa41a4948ba46209085f4ca00 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 06:55:55.864360 31546 raft_consensus.cc:3037] T 00000000000000000000000000000000 P dffab2aaa41a4948ba46209085f4ca00 [term 0 FOLLOWER]: Advancing to term 1
I20260812 06:55:55.865437 31546 raft_consensus.cc:504] T 00000000000000000000000000000000 P dffab2aaa41a4948ba46209085f4ca00 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dffab2aaa41a4948ba46209085f4ca00" member_type: VOTER last_known_addr { host: "127.19.52.126" port: 42435 } }
I20260812 06:55:55.865820 31546 leader_election.cc:302] T 00000000000000000000000000000000 P dffab2aaa41a4948ba46209085f4ca00 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: dffab2aaa41a4948ba46209085f4ca00; no voters:
I20260812 06:55:55.866237 31546 leader_election.cc:288] T 00000000000000000000000000000000 P dffab2aaa41a4948ba46209085f4ca00 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 06:55:55.866318 31551 raft_consensus.cc:2781] T 00000000000000000000000000000000 P dffab2aaa41a4948ba46209085f4ca00 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 06:55:55.866515 31551 raft_consensus.cc:686] T 00000000000000000000000000000000 P dffab2aaa41a4948ba46209085f4ca00 [term 1 LEADER]: Becoming Leader. State: Replica: dffab2aaa41a4948ba46209085f4ca00, State: Running, Role: LEADER
I20260812 06:55:55.866894 31551 consensus_queue.cc:237] T 00000000000000000000000000000000 P dffab2aaa41a4948ba46209085f4ca00 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dffab2aaa41a4948ba46209085f4ca00" member_type: VOTER last_known_addr { host: "127.19.52.126" port: 42435 } }
I20260812 06:55:55.867674 31546 sys_catalog.cc:564] T 00000000000000000000000000000000 P dffab2aaa41a4948ba46209085f4ca00 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 06:55:55.869138 31552 sys_catalog.cc:455] T 00000000000000000000000000000000 P dffab2aaa41a4948ba46209085f4ca00 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "dffab2aaa41a4948ba46209085f4ca00" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dffab2aaa41a4948ba46209085f4ca00" member_type: VOTER last_known_addr { host: "127.19.52.126" port: 42435 } } }
I20260812 06:55:55.869612 31552 sys_catalog.cc:458] T 00000000000000000000000000000000 P dffab2aaa41a4948ba46209085f4ca00 [sys.catalog]: This master's current role is: LEADER
I20260812 06:55:55.869495 31553 sys_catalog.cc:455] T 00000000000000000000000000000000 P dffab2aaa41a4948ba46209085f4ca00 [sys.catalog]: SysCatalogTable state changed. Reason: New leader dffab2aaa41a4948ba46209085f4ca00. Latest consensus state: current_term: 1 leader_uuid: "dffab2aaa41a4948ba46209085f4ca00" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dffab2aaa41a4948ba46209085f4ca00" member_type: VOTER last_known_addr { host: "127.19.52.126" port: 42435 } } }
I20260812 06:55:55.869961 31553 sys_catalog.cc:458] T 00000000000000000000000000000000 P dffab2aaa41a4948ba46209085f4ca00 [sys.catalog]: This master's current role is: LEADER
I20260812 06:55:55.870159 31549 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:55.854115 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51512 (local address 127.19.52.126:42435)
0812 06:55:55.854553 (+ 438us) server_negotiation.cc:206] Beginning negotiation
0812 06:55:55.854570 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0812 06:55:55.854609 (+ 39us) server_negotiation.cc:407] Connection header received
0812 06:55:55.855365 (+ 756us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:55:55.855383 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:55:55.855819 (+ 436us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:55:55.856237 (+ 418us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:55:55.857384 (+ 1147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:55.858495 (+ 1111us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:55.859699 (+ 1204us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:55.860184 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:55.860306 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:55:55.864049 (+ 3743us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:55:55.864054 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:55:55.864074 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:55:55.864110 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:55:55.866257 (+ 2147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:55:55.866883 (+ 626us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:55:55.866891 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:55:55.866900 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:55:55.867041 (+ 141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:55:55.867360 (+ 319us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:55:55.867363 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:55:55.867364 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:55:55.867814 (+ 450us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:55:55.868051 (+ 237us) server_negotiation.cc:1035] Waiting for connection context
0812 06:55:55.868373 (+ 322us) server_negotiation.cc:299] Negotiation successful
0812 06:55:55.868778 (+ 405us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":285,"thread_start_us":150,"threads_started":1}
W20260812 06:55:55.873113 31566 catalog_manager.cc:1558] T 00000000000000000000000000000000 P dffab2aaa41a4948ba46209085f4ca00: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 06:55:55.873188 31566 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 06:55:55.873267 31565 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 06:55:55.873996 31565 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 06:55:55.879721 31565 catalog_manager.cc:1347] Generated new cluster ID: 324e6f7ca16d4ac09cabc05032e2db41
I20260812 06:55:55.879813 31565 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 06:55:55.888361 31565 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 06:55:55.889683 31565 catalog_manager.cc:1504] Loading token signing keys...
I20260812 06:55:55.900983 31565 catalog_manager.cc:5868] T 00000000000000000000000000000000 P dffab2aaa41a4948ba46209085f4ca00: Generated new TSK 0
I20260812 06:55:55.901933 31565 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.19.52.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.52.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.52.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786517687384214-19665-0/krb5kdc/kudu_127.19.52.65.keytab.
Entry for principal kudu/127.19.52.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786517687384214-19665-0/krb5kdc/kudu_127.19.52.65.keytab.
I20260812 06:55:55.932405 19665 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu
/tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786517687384214-19665-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786517687384214-19665-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786517687384214-19665-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786517687384214-19665-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.52.65:0
--local_ip_for_outbound_sockets=127.19.52.65
--webserver_interface=127.19.52.65
--webserver_port=0
--tserver_master_addrs=127.19.52.126:42435
--keytab_file=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786517687384214-19665-0/krb5kdc/kudu_127.19.52.65.keytab
--principal=kudu/127.19.52.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.52.84:40101
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786517687384214-19665-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786517687384214-19665-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786517687384214-19665-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 06:55:56.053912 31572 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:55:56.054221 31572 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:55:56.054270 31572 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 06:55:56.058581 31572 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 06:55:56.058667 31572 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 06:55:56.058730 31572 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.52.65
I20260812 06:55:56.064729 31572 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.52.84:40101
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786517687384214-19665-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786517687384214-19665-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786517687384214-19665-0/krb5kdc/kudu_127.19.52.65.keytab
--principal=kudu/127.19.52.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.52.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786517687384214-19665-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.19.52.65
--webserver_port=0
--tserver_master_addrs=127.19.52.126:42435
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31572
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.52.65
--log_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786517687384214-19665-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 06:15:54 UTC on ccycloud.quasar-rnfzce.root.comops.site
build id 4581989
I20260812 06:55:56.065975 31572 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:55:56.067067 31572 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 06:55:56.078799 31577 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:55:56.078799 31578 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:55:56.078804 31580 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:55:56.079397 31572 server_base.cc:1034] running on GCE node
I20260812 06:55:56.079887 31572 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:55:56.080742 31572 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:55:56.081952 31572 hybrid_clock.cc:648] HybridClock initialized: now 1786517756081925 us; error 38 us; skew 500 ppm
Aug 12 06:55:56 dist-test-slave-65mx krb5kdc[31453](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517756, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.52.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 06:55:56.085282 31572 init.cc:377] Logged in from keytab as kudu/127.19.52.65@KRBTEST.COM (short username kudu)
I20260812 06:55:56.086393 31572 webserver.cc:460] Webserver started at http://127.19.52.65:38239/ using document root <none> and password file <none>
I20260812 06:55:56.087003 31572 fs_manager.cc:359] Metadata directory not provided
I20260812 06:55:56.087112 31572 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:55:56.087364 31572 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 06:55:56.089612 31572 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786517687384214-19665-0/minicluster-data/ts-0/data/instance:
uuid: "01e9f5bd855b4f5b9abdc45a16316287"
format_stamp: "Formatted at 2026-08-12 06:55:56 on dist-test-slave-65mx"
server_key: "52fb7442a45a61894725407d2b5f4cb2"
server_key_iv: "56f0f57074052a0236abbecd83776a0f"
server_key_version: "encryptionkey@0"
I20260812 06:55:56.090204 31572 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786517687384214-19665-0/minicluster-data/ts-0/wal/instance:
uuid: "01e9f5bd855b4f5b9abdc45a16316287"
format_stamp: "Formatted at 2026-08-12 06:55:56 on dist-test-slave-65mx"
server_key: "52fb7442a45a61894725407d2b5f4cb2"
server_key_iv: "56f0f57074052a0236abbecd83776a0f"
server_key_version: "encryptionkey@0"
I20260812 06:55:56.094147 31572 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260812 06:55:56.096807 31588 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 06:55:56.098201 31572 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 06:55:56.098378 31572 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786517687384214-19665-0/minicluster-data/ts-0/data,/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786517687384214-19665-0/minicluster-data/ts-0/wal
uuid: "01e9f5bd855b4f5b9abdc45a16316287"
format_stamp: "Formatted at 2026-08-12 06:55:56 on dist-test-slave-65mx"
server_key: "52fb7442a45a61894725407d2b5f4cb2"
server_key_iv: "56f0f57074052a0236abbecd83776a0f"
server_key_version: "encryptionkey@0"
I20260812 06:55:56.098513 31572 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786517687384214-19665-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786517687384214-19665-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786517687384214-19665-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:55:56.118427 31572 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 06:55:56.122140 31572 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 06:55:56.122363 31572 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:55:56.123003 31572 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 06:55:56.123862 31572 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 06:55:56.123944 31572 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 06:55:56.124018 31572 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 06:55:56.124073 31572 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 06:55:56.133935 31572 rpc_server.cc:307] RPC server started. Bound to: 127.19.52.65:36181
I20260812 06:55:56.133945 31712 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.52.65:36181 every 8 connection(s)
I20260812 06:55:56.134888 31572 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786517687384214-19665-0/minicluster-data/ts-0/data/info.pb
I20260812 06:55:56.140908 19665 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu as pid 31572
I20260812 06:55:56.141057 19665 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786517687384214-19665-0/minicluster-data/ts-0/wal/instance
I20260812 06:55:56.141340 19665 external_mini_cluster.cc:1428] Setting key 78d15e688e704ba36d0f6a5701756698
Aug 12 06:55:56 dist-test-slave-65mx krb5kdc[31453](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517756, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.52.65@KRBTEST.COM for kudu/127.19.52.126@KRBTEST.COM
I20260812 06:55:56.152089 31549 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:56.136819 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.52.65:40205 (local address 127.19.52.126:42435)
0812 06:55:56.137018 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0812 06:55:56.137023 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 06:55:56.137934 (+ 911us) server_negotiation.cc:407] Connection header received
0812 06:55:56.139352 (+ 1418us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:55:56.139355 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:55:56.139406 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:55:56.139521 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:55:56.140907 (+ 1386us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:56.141682 (+ 775us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:56.143633 (+ 1951us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:56.143881 (+ 248us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:56.143980 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:55:56.147388 (+ 3408us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:55:56.147391 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:55:56.147393 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:55:56.147420 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:55:56.149313 (+ 1893us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:55:56.150100 (+ 787us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:55:56.150103 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:55:56.150104 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:55:56.150196 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:55:56.150701 (+ 505us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:55:56.150704 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:55:56.150705 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:55:56.150950 (+ 245us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:55:56.151051 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0812 06:55:56.151776 (+ 725us) server_negotiation.cc:299] Negotiation successful
0812 06:55:56.151951 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260812 06:55:56.153388 31715 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:56.137270 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.126:42435 (local address 127.19.52.65:40205)
0812 06:55:56.137756 (+ 486us) negotiation.cc:107] Waiting for socket to connect
0812 06:55:56.137804 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0812 06:55:56.139114 (+ 1310us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:55:56.139708 (+ 594us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:55:56.139715 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:55:56.140122 (+ 407us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:55:56.140738 (+ 616us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:55:56.140755 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:55:56.142119 (+ 1364us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:55:56.142123 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:55:56.143331 (+ 1208us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:55:56.143345 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:55:56.144114 (+ 769us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:55:56.144121 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:55:56.144389 (+ 268us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:55:56.145136 (+ 747us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 06:55:56.145164 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:55:56.147169 (+ 2005us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 06:55:56.149469 (+ 2300us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:55:56.149479 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:55:56.149499 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:55:56.149956 (+ 457us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:55:56.150325 (+ 369us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:55:56.150331 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:55:56.150336 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:55:56.150567 (+ 231us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:55:56.151075 (+ 508us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 06:55:56.151084 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 06:55:56.151466 (+ 382us) client_negotiation.cc:769] Sending connection context
0812 06:55:56.151715 (+ 249us) client_negotiation.cc:240] Negotiation successful
0812 06:55:56.152139 (+ 424us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":286,"thread_start_us":131,"threads_started":1}
WARNING: no policy specified for kudu/127.19.52.66@KRBTEST.COM; defaulting to no policy
I20260812 06:55:56.154837 31713 heartbeater.cc:347] Connected to a master server at 127.19.52.126:42435
I20260812 06:55:56.155151 31713 heartbeater.cc:464] Registering TS with master...
I20260812 06:55:56.155690 31713 heartbeater.cc:511] Master 127.19.52.126:42435 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.52.66@KRBTEST.COM" created.
I20260812 06:55:56.157685 31505 ts_manager.cc:194] Registered new tserver with Master: 01e9f5bd855b4f5b9abdc45a16316287 (127.19.52.65:36181)
I20260812 06:55:56.158948 31505 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.52.65@KRBTEST.COM'} at 127.19.52.65:40205
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.52.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786517687384214-19665-0/krb5kdc/kudu_127.19.52.66.keytab.
Entry for principal kudu/127.19.52.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786517687384214-19665-0/krb5kdc/kudu_127.19.52.66.keytab.
I20260812 06:55:56.176839 19665 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu
/tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786517687384214-19665-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786517687384214-19665-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786517687384214-19665-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786517687384214-19665-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.52.66:0
--local_ip_for_outbound_sockets=127.19.52.66
--webserver_interface=127.19.52.66
--webserver_port=0
--tserver_master_addrs=127.19.52.126:42435
--keytab_file=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786517687384214-19665-0/krb5kdc/kudu_127.19.52.66.keytab
--principal=kudu/127.19.52.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.52.84:40101
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786517687384214-19665-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786517687384214-19665-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786517687384214-19665-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 06:55:56.300076 31718 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:55:56.300376 31718 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:55:56.300421 31718 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 06:55:56.305014 31718 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 06:55:56.305083 31718 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 06:55:56.305145 31718 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.52.66
I20260812 06:55:56.310662 31718 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.52.84:40101
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786517687384214-19665-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786517687384214-19665-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786517687384214-19665-0/krb5kdc/kudu_127.19.52.66.keytab
--principal=kudu/127.19.52.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.52.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786517687384214-19665-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.19.52.66
--webserver_port=0
--tserver_master_addrs=127.19.52.126:42435
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31718
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.52.66
--log_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786517687384214-19665-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 06:15:54 UTC on ccycloud.quasar-rnfzce.root.comops.site
build id 4581989
I20260812 06:55:56.311793 31718 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:55:56.312820 31718 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 06:55:56.323151 31727 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:55:56.323130 31725 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:55:56.323112 31724 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:55:56.323838 31718 server_base.cc:1034] running on GCE node
I20260812 06:55:56.324425 31718 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:55:56.325199 31718 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:55:56.326443 31718 hybrid_clock.cc:648] HybridClock initialized: now 1786517756326426 us; error 48 us; skew 500 ppm
Aug 12 06:55:56 dist-test-slave-65mx krb5kdc[31453](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517756, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.52.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 06:55:56.329648 31718 init.cc:377] Logged in from keytab as kudu/127.19.52.66@KRBTEST.COM (short username kudu)
I20260812 06:55:56.330669 31718 webserver.cc:460] Webserver started at http://127.19.52.66:46009/ using document root <none> and password file <none>
I20260812 06:55:56.331373 31718 fs_manager.cc:359] Metadata directory not provided
I20260812 06:55:56.331477 31718 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:55:56.331732 31718 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 06:55:56.334039 31718 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786517687384214-19665-0/minicluster-data/ts-1/data/instance:
uuid: "a0155d1ee3c14b3cbdd171d2d606d144"
format_stamp: "Formatted at 2026-08-12 06:55:56 on dist-test-slave-65mx"
server_key: "916cc87506b4c7d7e909de35f75b09c2"
server_key_iv: "67867c3bb8ed15770366a2c74fecd1a5"
server_key_version: "encryptionkey@0"
I20260812 06:55:56.334669 31718 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786517687384214-19665-0/minicluster-data/ts-1/wal/instance:
uuid: "a0155d1ee3c14b3cbdd171d2d606d144"
format_stamp: "Formatted at 2026-08-12 06:55:56 on dist-test-slave-65mx"
server_key: "916cc87506b4c7d7e909de35f75b09c2"
server_key_iv: "67867c3bb8ed15770366a2c74fecd1a5"
server_key_version: "encryptionkey@0"
I20260812 06:55:56.338925 31718 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260812 06:55:56.341602 31736 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 06:55:56.342998 31718 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 06:55:56.343161 31718 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786517687384214-19665-0/minicluster-data/ts-1/data,/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786517687384214-19665-0/minicluster-data/ts-1/wal
uuid: "a0155d1ee3c14b3cbdd171d2d606d144"
format_stamp: "Formatted at 2026-08-12 06:55:56 on dist-test-slave-65mx"
server_key: "916cc87506b4c7d7e909de35f75b09c2"
server_key_iv: "67867c3bb8ed15770366a2c74fecd1a5"
server_key_version: "encryptionkey@0"
I20260812 06:55:56.343286 31718 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786517687384214-19665-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786517687384214-19665-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786517687384214-19665-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:55:56.361604 31718 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 06:55:56.365471 31718 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 06:55:56.365726 31718 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:55:56.366411 31718 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 06:55:56.367329 31718 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 06:55:56.367408 31718 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 06:55:56.367486 31718 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 06:55:56.367537 31718 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 06:55:56.379446 31718 rpc_server.cc:307] RPC server started. Bound to: 127.19.52.66:33431
I20260812 06:55:56.379472 31858 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.52.66:33431 every 8 connection(s)
I20260812 06:55:56.380470 31718 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786517687384214-19665-0/minicluster-data/ts-1/data/info.pb
I20260812 06:55:56.385694 19665 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu as pid 31718
I20260812 06:55:56.385836 19665 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786517687384214-19665-0/minicluster-data/ts-1/wal/instance
I20260812 06:55:56.386163 19665 external_mini_cluster.cc:1428] Setting key bb46e25f2c9eedfdc323f41fdd7123e8
Aug 12 06:55:56 dist-test-slave-65mx krb5kdc[31453](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517756, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.52.66@KRBTEST.COM for kudu/127.19.52.126@KRBTEST.COM
I20260812 06:55:56.396086 31549 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:56.382651 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.52.66:47493 (local address 127.19.52.126:42435)
0812 06:55:56.382841 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0812 06:55:56.382846 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 06:55:56.384042 (+ 1196us) server_negotiation.cc:407] Connection header received
0812 06:55:56.385583 (+ 1541us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:55:56.385587 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:55:56.385671 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:55:56.385792 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:55:56.387322 (+ 1530us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:56.387966 (+ 644us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:56.389288 (+ 1322us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:56.389517 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:56.389639 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:55:56.392294 (+ 2655us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:55:56.392297 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:55:56.392300 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:55:56.392327 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:55:56.393896 (+ 1569us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:55:56.394460 (+ 564us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:55:56.394464 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:55:56.394466 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:55:56.394554 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:55:56.394926 (+ 372us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:55:56.394929 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:55:56.394931 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:55:56.395134 (+ 203us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:55:56.395244 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0812 06:55:56.395760 (+ 516us) server_negotiation.cc:299] Negotiation successful
0812 06:55:56.395896 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":80}
WARNING: no policy specified for kudu/127.19.52.67@KRBTEST.COM; defaulting to no policy
I20260812 06:55:56.397092 31861 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:56.383180 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.126:42435 (local address 127.19.52.66:47493)
0812 06:55:56.383842 (+ 662us) negotiation.cc:107] Waiting for socket to connect
0812 06:55:56.383890 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0812 06:55:56.385250 (+ 1360us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:55:56.386039 (+ 789us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:55:56.386051 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:55:56.386510 (+ 459us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:55:56.387153 (+ 643us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:55:56.387169 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:55:56.388112 (+ 943us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:55:56.388117 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:55:56.389145 (+ 1028us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:55:56.389156 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:55:56.389694 (+ 538us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:55:56.389698 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:55:56.389881 (+ 183us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:55:56.390562 (+ 681us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 06:55:56.390585 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:55:56.392094 (+ 1509us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 06:55:56.394037 (+ 1943us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:55:56.394043 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:55:56.394059 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:55:56.394346 (+ 287us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:55:56.394655 (+ 309us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:55:56.394659 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:55:56.394661 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:55:56.394817 (+ 156us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:55:56.395245 (+ 428us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 06:55:56.395252 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 06:55:56.395513 (+ 261us) client_negotiation.cc:769] Sending connection context
0812 06:55:56.395717 (+ 204us) client_negotiation.cc:240] Negotiation successful
0812 06:55:56.396067 (+ 350us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":455,"thread_start_us":257,"threads_started":1}
I20260812 06:55:56.398402 31859 heartbeater.cc:347] Connected to a master server at 127.19.52.126:42435
I20260812 06:55:56.398663 31859 heartbeater.cc:464] Registering TS with master...
I20260812 06:55:56.399210 31859 heartbeater.cc:511] Master 127.19.52.126:42435 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.52.67@KRBTEST.COM" created.
I20260812 06:55:56.400367 31505 ts_manager.cc:194] Registered new tserver with Master: a0155d1ee3c14b3cbdd171d2d606d144 (127.19.52.66:33431)
I20260812 06:55:56.401098 31505 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.52.66@KRBTEST.COM'} at 127.19.52.66:47493
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.52.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786517687384214-19665-0/krb5kdc/kudu_127.19.52.67.keytab.
Entry for principal kudu/127.19.52.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786517687384214-19665-0/krb5kdc/kudu_127.19.52.67.keytab.
I20260812 06:55:56.420221 19665 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu
/tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786517687384214-19665-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786517687384214-19665-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786517687384214-19665-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786517687384214-19665-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.52.67:0
--local_ip_for_outbound_sockets=127.19.52.67
--webserver_interface=127.19.52.67
--webserver_port=0
--tserver_master_addrs=127.19.52.126:42435
--keytab_file=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786517687384214-19665-0/krb5kdc/kudu_127.19.52.67.keytab
--principal=kudu/127.19.52.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.52.84:40101
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786517687384214-19665-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786517687384214-19665-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786517687384214-19665-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 06:55:56.536684 31864 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:55:56.537005 31864 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:55:56.537117 31864 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 06:55:56.541204 31864 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 06:55:56.541303 31864 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 06:55:56.541399 31864 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.52.67
I20260812 06:55:56.546867 31864 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.52.84:40101
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786517687384214-19665-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786517687384214-19665-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786517687384214-19665-0/krb5kdc/kudu_127.19.52.67.keytab
--principal=kudu/127.19.52.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.52.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786517687384214-19665-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.19.52.67
--webserver_port=0
--tserver_master_addrs=127.19.52.126:42435
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31864
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.52.67
--log_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786517687384214-19665-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 06:15:54 UTC on ccycloud.quasar-rnfzce.root.comops.site
build id 4581989
I20260812 06:55:56.548023 31864 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:55:56.549105 31864 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 06:55:56.559262 31872 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:55:56.559258 31869 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:55:56.559253 31870 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:55:56.559629 31864 server_base.cc:1034] running on GCE node
I20260812 06:55:56.560246 31864 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:55:56.561156 31864 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:55:56.562366 31864 hybrid_clock.cc:648] HybridClock initialized: now 1786517756562377 us; error 90 us; skew 500 ppm
Aug 12 06:55:56 dist-test-slave-65mx krb5kdc[31453](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517756, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.52.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 06:55:56.565630 31864 init.cc:377] Logged in from keytab as kudu/127.19.52.67@KRBTEST.COM (short username kudu)
I20260812 06:55:56.566545 31864 webserver.cc:460] Webserver started at http://127.19.52.67:41763/ using document root <none> and password file <none>
I20260812 06:55:56.567104 31864 fs_manager.cc:359] Metadata directory not provided
I20260812 06:55:56.567184 31864 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:55:56.567417 31864 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 06:55:56.569762 31864 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786517687384214-19665-0/minicluster-data/ts-2/data/instance:
uuid: "d6365d41e5504beba5648b3fe514eaee"
format_stamp: "Formatted at 2026-08-12 06:55:56 on dist-test-slave-65mx"
server_key: "3eb97860320e5eb68a3648a00685a0a5"
server_key_iv: "62a2e6d9cfac11f97e0e7893f54d30b9"
server_key_version: "encryptionkey@0"
I20260812 06:55:56.570369 31864 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786517687384214-19665-0/minicluster-data/ts-2/wal/instance:
uuid: "d6365d41e5504beba5648b3fe514eaee"
format_stamp: "Formatted at 2026-08-12 06:55:56 on dist-test-slave-65mx"
server_key: "3eb97860320e5eb68a3648a00685a0a5"
server_key_iv: "62a2e6d9cfac11f97e0e7893f54d30b9"
server_key_version: "encryptionkey@0"
I20260812 06:55:56.574481 31864 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.004s
I20260812 06:55:56.577144 31880 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 06:55:56.578526 31864 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 06:55:56.578632 31864 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786517687384214-19665-0/minicluster-data/ts-2/data,/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786517687384214-19665-0/minicluster-data/ts-2/wal
uuid: "d6365d41e5504beba5648b3fe514eaee"
format_stamp: "Formatted at 2026-08-12 06:55:56 on dist-test-slave-65mx"
server_key: "3eb97860320e5eb68a3648a00685a0a5"
server_key_iv: "62a2e6d9cfac11f97e0e7893f54d30b9"
server_key_version: "encryptionkey@0"
I20260812 06:55:56.578729 31864 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786517687384214-19665-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786517687384214-19665-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786517687384214-19665-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:55:56.606094 31864 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 06:55:56.609596 31864 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 06:55:56.609767 31864 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:55:56.610333 31864 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 06:55:56.611142 31864 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 06:55:56.611191 31864 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 06:55:56.611230 31864 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 06:55:56.611246 31864 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 06:55:56.620108 31864 rpc_server.cc:307] RPC server started. Bound to: 127.19.52.67:39047
I20260812 06:55:56.620139 32006 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.52.67:39047 every 8 connection(s)
I20260812 06:55:56.621066 31864 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786517687384214-19665-0/minicluster-data/ts-2/data/info.pb
I20260812 06:55:56.629539 19665 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu as pid 31864
I20260812 06:55:56.629714 19665 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786517687384214-19665-0/minicluster-data/ts-2/wal/instance
I20260812 06:55:56.630055 19665 external_mini_cluster.cc:1428] Setting key 1493524a1824749ca01c628a2caf8a8f
Aug 12 06:55:56 dist-test-slave-65mx krb5kdc[31453](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517756, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.52.67@KRBTEST.COM for kudu/127.19.52.126@KRBTEST.COM
I20260812 06:55:56.635468 31549 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:56.622884 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.52.67:37473 (local address 127.19.52.126:42435)
0812 06:55:56.623018 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0812 06:55:56.623022 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 06:55:56.623817 (+ 795us) server_negotiation.cc:407] Connection header received
0812 06:55:56.625328 (+ 1511us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:55:56.625332 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:55:56.625394 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:55:56.625474 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:55:56.626666 (+ 1192us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:56.627213 (+ 547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:56.628375 (+ 1162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:56.628701 (+ 326us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:56.628820 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:55:56.631804 (+ 2984us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:55:56.631807 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:55:56.631810 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:55:56.631837 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:55:56.633301 (+ 1464us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:55:56.633860 (+ 559us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:55:56.633870 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:55:56.633872 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:55:56.633951 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:55:56.634306 (+ 355us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:55:56.634309 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:55:56.634311 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:55:56.634515 (+ 204us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:55:56.634626 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0812 06:55:56.635148 (+ 522us) server_negotiation.cc:299] Negotiation successful
0812 06:55:56.635280 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260812 06:55:56.636315 32009 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:56.623227 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.126:42435 (local address 127.19.52.67:37473)
0812 06:55:56.623666 (+ 439us) negotiation.cc:107] Waiting for socket to connect
0812 06:55:56.623709 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0812 06:55:56.625112 (+ 1403us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:55:56.625639 (+ 527us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:55:56.625645 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:55:56.626009 (+ 364us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:55:56.626507 (+ 498us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:55:56.626520 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:55:56.627355 (+ 835us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:55:56.627358 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:55:56.628229 (+ 871us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:55:56.628238 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:55:56.628862 (+ 624us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:55:56.628866 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:55:56.629038 (+ 172us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:55:56.629712 (+ 674us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 06:55:56.629737 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:55:56.631619 (+ 1882us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 06:55:56.633434 (+ 1815us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:55:56.633440 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:55:56.633457 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:55:56.633738 (+ 281us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:55:56.634050 (+ 312us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:55:56.634053 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:55:56.634055 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:55:56.634202 (+ 147us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:55:56.634626 (+ 424us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 06:55:56.634635 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 06:55:56.634895 (+ 260us) client_negotiation.cc:769] Sending connection context
0812 06:55:56.635103 (+ 208us) client_negotiation.cc:240] Negotiation successful
0812 06:55:56.635460 (+ 357us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":257,"thread_start_us":117,"threads_started":1}
I20260812 06:55:56.637631 32007 heartbeater.cc:347] Connected to a master server at 127.19.52.126:42435
I20260812 06:55:56.637920 32007 heartbeater.cc:464] Registering TS with master...
I20260812 06:55:56.638471 32007 heartbeater.cc:511] Master 127.19.52.126:42435 requested a full tablet report, sending...
I20260812 06:55:56.639637 31505 ts_manager.cc:194] Registered new tserver with Master: d6365d41e5504beba5648b3fe514eaee (127.19.52.67:39047)
I20260812 06:55:56.640240 31505 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.52.67@KRBTEST.COM'} at 127.19.52.67:37473
I20260812 06:55:56.644126 19665 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 06:55:56.654537 31549 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:56.646164 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51516 (local address 127.19.52.126:42435)
0812 06:55:56.646333 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0812 06:55:56.646338 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 06:55:56.646519 (+ 181us) server_negotiation.cc:407] Connection header received
0812 06:55:56.646618 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:55:56.646620 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:55:56.646661 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:55:56.646729 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:55:56.647428 (+ 699us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:56.648070 (+ 642us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:56.648986 (+ 916us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:56.649199 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:56.649296 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:55:56.650777 (+ 1481us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:55:56.650780 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:55:56.650782 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:55:56.650815 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:55:56.652231 (+ 1416us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:55:56.652799 (+ 568us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:55:56.652802 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:55:56.652803 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:55:56.652887 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:55:56.653222 (+ 335us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:55:56.653225 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:55:56.653226 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:55:56.653625 (+ 399us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:55:56.653747 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0812 06:55:56.654088 (+ 341us) server_negotiation.cc:299] Negotiation successful
0812 06:55:56.654335 (+ 247us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260812 06:55:56.657724 31505 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51516:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 06:55:56.660918 31505 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 06:55:56.677026 32023 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:56.670559 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35288 (local address 127.19.52.65:36181)
0812 06:55:56.672114 (+ 1555us) server_negotiation.cc:206] Beginning negotiation
0812 06:55:56.672120 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 06:55:56.672145 (+ 25us) server_negotiation.cc:407] Connection header received
0812 06:55:56.672216 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:55:56.672220 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:55:56.672387 (+ 167us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:55:56.672530 (+ 143us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 06:55:56.672821 (+ 291us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:56.673869 (+ 1048us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:56.675183 (+ 1314us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:56.675927 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:56.676028 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:55:56.676184 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0812 06:55:56.676717 (+ 533us) server_negotiation.cc:299] Negotiation successful
0812 06:55:56.676828 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1436,"thread_start_us":84,"threads_started":1}
I20260812 06:55:56.677032 32021 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:56.669943 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.65:36181 (local address 127.0.0.1:35288)
0812 06:55:56.670984 (+ 1041us) negotiation.cc:107] Waiting for socket to connect
0812 06:55:56.671037 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0812 06:55:56.671189 (+ 152us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:55:56.672571 (+ 1382us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:55:56.672574 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:55:56.672594 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 06:55:56.672694 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:55:56.672700 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:55:56.673973 (+ 1273us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:55:56.673976 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:55:56.675065 (+ 1089us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:55:56.675076 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:55:56.676255 (+ 1179us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:55:56.676259 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:55:56.676436 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:55:56.676513 (+ 77us) client_negotiation.cc:769] Sending connection context
0812 06:55:56.676660 (+ 147us) client_negotiation.cc:240] Negotiation successful
0812 06:55:56.676853 (+ 193us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":944,"mutex_wait_us":678,"spinlock_wait_cycles":30720,"thread_start_us":101,"threads_started":1}
I20260812 06:55:56.677418 32024 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:56.669940 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.66:33431 (local address 127.0.0.1:43022)
0812 06:55:56.670985 (+ 1045us) negotiation.cc:107] Waiting for socket to connect
0812 06:55:56.671038 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0812 06:55:56.671189 (+ 151us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:55:56.671774 (+ 585us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:55:56.671779 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:55:56.671802 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 06:55:56.671951 (+ 149us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:55:56.671960 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:55:56.673460 (+ 1500us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:55:56.673463 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:55:56.674888 (+ 1425us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:55:56.674898 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:55:56.676255 (+ 1357us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:55:56.676259 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:55:56.676439 (+ 180us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:55:56.676513 (+ 74us) client_negotiation.cc:769] Sending connection context
0812 06:55:56.676660 (+ 147us) client_negotiation.cc:240] Negotiation successful
0812 06:55:56.677260 (+ 600us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":870,"mutex_wait_us":549,"spinlock_wait_cycles":16640,"thread_start_us":95,"threads_started":1}
I20260812 06:55:56.677670 32019 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:56.670212 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.67:39047 (local address 127.0.0.1:34422)
0812 06:55:56.670985 (+ 773us) negotiation.cc:107] Waiting for socket to connect
0812 06:55:56.671038 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0812 06:55:56.671189 (+ 151us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:55:56.671891 (+ 702us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:55:56.671895 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:55:56.671913 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 06:55:56.672038 (+ 125us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:55:56.672049 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:55:56.673292 (+ 1243us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:55:56.673298 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:55:56.675817 (+ 2519us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:55:56.675828 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:55:56.676976 (+ 1148us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:55:56.676979 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:55:56.677112 (+ 133us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:55:56.677376 (+ 264us) client_negotiation.cc:769] Sending connection context
0812 06:55:56.677458 (+ 82us) client_negotiation.cc:240] Negotiation successful
0812 06:55:56.677538 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":609,"mutex_wait_us":1322,"spinlock_wait_cycles":46208,"thread_start_us":125,"threads_started":1}
I20260812 06:55:56.677795 32020 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:56.670123 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43022 (local address 127.19.52.66:33431)
0812 06:55:56.670802 (+ 679us) server_negotiation.cc:206] Beginning negotiation
0812 06:55:56.670815 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0812 06:55:56.671339 (+ 524us) server_negotiation.cc:407] Connection header received
0812 06:55:56.671428 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:55:56.671434 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:55:56.671611 (+ 177us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:55:56.671779 (+ 168us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 06:55:56.672090 (+ 311us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:56.673315 (+ 1225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:56.675025 (+ 1710us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:56.675913 (+ 888us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:56.676028 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:55:56.676184 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0812 06:55:56.677521 (+ 1337us) server_negotiation.cc:299] Negotiation successful
0812 06:55:56.677625 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":568,"thread_start_us":153,"threads_started":1}
I20260812 06:55:56.678205 32022 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:56.670464 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34422 (local address 127.19.52.67:39047)
0812 06:55:56.671521 (+ 1057us) server_negotiation.cc:206] Beginning negotiation
0812 06:55:56.671532 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0812 06:55:56.671560 (+ 28us) server_negotiation.cc:407] Connection header received
0812 06:55:56.671621 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:55:56.671625 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:55:56.671759 (+ 134us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:55:56.671912 (+ 153us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 06:55:56.672141 (+ 229us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:56.673159 (+ 1018us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:56.675974 (+ 2815us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:56.676622 (+ 648us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:56.676753 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:55:56.676888 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0812 06:55:56.677930 (+ 1042us) server_negotiation.cc:299] Negotiation successful
0812 06:55:56.678055 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":924,"thread_start_us":172,"threads_started":1}
I20260812 06:55:56.681056 31933 tablet_service.cc:1467] Processing CreateTablet for tablet 90de5c43799e4ab288213e1c56a0e80a (DEFAULT_TABLE table=test-table [id=80205dfabf2a47b3b6344e1c7c02e583]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 06:55:56.681061 31787 tablet_service.cc:1467] Processing CreateTablet for tablet 90de5c43799e4ab288213e1c56a0e80a (DEFAULT_TABLE table=test-table [id=80205dfabf2a47b3b6344e1c7c02e583]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 06:55:56.681053 31642 tablet_service.cc:1467] Processing CreateTablet for tablet 90de5c43799e4ab288213e1c56a0e80a (DEFAULT_TABLE table=test-table [id=80205dfabf2a47b3b6344e1c7c02e583]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 06:55:56.682240 31642 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 90de5c43799e4ab288213e1c56a0e80a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 06:55:56.682248 31787 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 90de5c43799e4ab288213e1c56a0e80a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 06:55:56.682255 31933 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 90de5c43799e4ab288213e1c56a0e80a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 06:55:56.688892 32025 tablet_bootstrap.cc:492] T 90de5c43799e4ab288213e1c56a0e80a P 01e9f5bd855b4f5b9abdc45a16316287: Bootstrap starting.
I20260812 06:55:56.690328 32026 tablet_bootstrap.cc:492] T 90de5c43799e4ab288213e1c56a0e80a P d6365d41e5504beba5648b3fe514eaee: Bootstrap starting.
I20260812 06:55:56.690547 32028 tablet_bootstrap.cc:492] T 90de5c43799e4ab288213e1c56a0e80a P a0155d1ee3c14b3cbdd171d2d606d144: Bootstrap starting.
I20260812 06:55:56.691376 32025 tablet_bootstrap.cc:654] T 90de5c43799e4ab288213e1c56a0e80a P 01e9f5bd855b4f5b9abdc45a16316287: Neither blocks nor log segments found. Creating new log.
I20260812 06:55:56.692461 32025 log.cc:824] T 90de5c43799e4ab288213e1c56a0e80a P 01e9f5bd855b4f5b9abdc45a16316287: Log is configured to *not* fsync() on all Append() calls
I20260812 06:55:56.692741 32026 tablet_bootstrap.cc:654] T 90de5c43799e4ab288213e1c56a0e80a P d6365d41e5504beba5648b3fe514eaee: Neither blocks nor log segments found. Creating new log.
I20260812 06:55:56.693259 32028 tablet_bootstrap.cc:654] T 90de5c43799e4ab288213e1c56a0e80a P a0155d1ee3c14b3cbdd171d2d606d144: Neither blocks nor log segments found. Creating new log.
I20260812 06:55:56.694111 32026 log.cc:824] T 90de5c43799e4ab288213e1c56a0e80a P d6365d41e5504beba5648b3fe514eaee: Log is configured to *not* fsync() on all Append() calls
I20260812 06:55:56.694111 32028 log.cc:824] T 90de5c43799e4ab288213e1c56a0e80a P a0155d1ee3c14b3cbdd171d2d606d144: Log is configured to *not* fsync() on all Append() calls
I20260812 06:55:56.694815 32025 tablet_bootstrap.cc:492] T 90de5c43799e4ab288213e1c56a0e80a P 01e9f5bd855b4f5b9abdc45a16316287: No bootstrap required, opened a new log
I20260812 06:55:56.695055 32025 ts_tablet_manager.cc:1397] T 90de5c43799e4ab288213e1c56a0e80a P 01e9f5bd855b4f5b9abdc45a16316287: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260812 06:55:56.695947 32026 tablet_bootstrap.cc:492] T 90de5c43799e4ab288213e1c56a0e80a P d6365d41e5504beba5648b3fe514eaee: No bootstrap required, opened a new log
I20260812 06:55:56.696157 32026 ts_tablet_manager.cc:1397] T 90de5c43799e4ab288213e1c56a0e80a P d6365d41e5504beba5648b3fe514eaee: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.004s
I20260812 06:55:56.698778 32025 raft_consensus.cc:348] T 90de5c43799e4ab288213e1c56a0e80a P 01e9f5bd855b4f5b9abdc45a16316287 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0155d1ee3c14b3cbdd171d2d606d144" member_type: VOTER last_known_addr { host: "127.19.52.66" port: 33431 } } peers { permanent_uuid: "01e9f5bd855b4f5b9abdc45a16316287" member_type: VOTER last_known_addr { host: "127.19.52.65" port: 36181 } } peers { permanent_uuid: "d6365d41e5504beba5648b3fe514eaee" member_type: VOTER last_known_addr { host: "127.19.52.67" port: 39047 } }
I20260812 06:55:56.699044 32026 raft_consensus.cc:348] T 90de5c43799e4ab288213e1c56a0e80a P d6365d41e5504beba5648b3fe514eaee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0155d1ee3c14b3cbdd171d2d606d144" member_type: VOTER last_known_addr { host: "127.19.52.66" port: 33431 } } peers { permanent_uuid: "01e9f5bd855b4f5b9abdc45a16316287" member_type: VOTER last_known_addr { host: "127.19.52.65" port: 36181 } } peers { permanent_uuid: "d6365d41e5504beba5648b3fe514eaee" member_type: VOTER last_known_addr { host: "127.19.52.67" port: 39047 } }
I20260812 06:55:56.699218 32025 raft_consensus.cc:374] T 90de5c43799e4ab288213e1c56a0e80a P 01e9f5bd855b4f5b9abdc45a16316287 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 06:55:56.699280 32025 raft_consensus.cc:729] T 90de5c43799e4ab288213e1c56a0e80a P 01e9f5bd855b4f5b9abdc45a16316287 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 01e9f5bd855b4f5b9abdc45a16316287, State: Initialized, Role: FOLLOWER
I20260812 06:55:56.699361 32026 raft_consensus.cc:374] T 90de5c43799e4ab288213e1c56a0e80a P d6365d41e5504beba5648b3fe514eaee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 06:55:56.699498 32026 raft_consensus.cc:729] T 90de5c43799e4ab288213e1c56a0e80a P d6365d41e5504beba5648b3fe514eaee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d6365d41e5504beba5648b3fe514eaee, State: Initialized, Role: FOLLOWER
I20260812 06:55:56.699863 32025 consensus_queue.cc:260] T 90de5c43799e4ab288213e1c56a0e80a P 01e9f5bd855b4f5b9abdc45a16316287 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0155d1ee3c14b3cbdd171d2d606d144" member_type: VOTER last_known_addr { host: "127.19.52.66" port: 33431 } } peers { permanent_uuid: "01e9f5bd855b4f5b9abdc45a16316287" member_type: VOTER last_known_addr { host: "127.19.52.65" port: 36181 } } peers { permanent_uuid: "d6365d41e5504beba5648b3fe514eaee" member_type: VOTER last_known_addr { host: "127.19.52.67" port: 39047 } }
I20260812 06:55:56.699985 32026 consensus_queue.cc:260] T 90de5c43799e4ab288213e1c56a0e80a P d6365d41e5504beba5648b3fe514eaee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0155d1ee3c14b3cbdd171d2d606d144" member_type: VOTER last_known_addr { host: "127.19.52.66" port: 33431 } } peers { permanent_uuid: "01e9f5bd855b4f5b9abdc45a16316287" member_type: VOTER last_known_addr { host: "127.19.52.65" port: 36181 } } peers { permanent_uuid: "d6365d41e5504beba5648b3fe514eaee" member_type: VOTER last_known_addr { host: "127.19.52.67" port: 39047 } }
I20260812 06:55:56.700668 31713 heartbeater.cc:503] Master 127.19.52.126:42435 was elected leader, sending a full tablet report...
I20260812 06:55:56.700691 32028 tablet_bootstrap.cc:492] T 90de5c43799e4ab288213e1c56a0e80a P a0155d1ee3c14b3cbdd171d2d606d144: No bootstrap required, opened a new log
I20260812 06:55:56.700879 32028 ts_tablet_manager.cc:1397] T 90de5c43799e4ab288213e1c56a0e80a P a0155d1ee3c14b3cbdd171d2d606d144: Time spent bootstrapping tablet: real 0.010s user 0.001s sys 0.004s
I20260812 06:55:56.701014 32025 ts_tablet_manager.cc:1428] T 90de5c43799e4ab288213e1c56a0e80a P 01e9f5bd855b4f5b9abdc45a16316287: Time spent starting tablet: real 0.006s user 0.000s sys 0.005s
I20260812 06:55:56.701057 32026 ts_tablet_manager.cc:1428] T 90de5c43799e4ab288213e1c56a0e80a P d6365d41e5504beba5648b3fe514eaee: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260812 06:55:56.701371 32007 heartbeater.cc:503] Master 127.19.52.126:42435 was elected leader, sending a full tablet report...
I20260812 06:55:56.704975 32028 raft_consensus.cc:348] T 90de5c43799e4ab288213e1c56a0e80a P a0155d1ee3c14b3cbdd171d2d606d144 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0155d1ee3c14b3cbdd171d2d606d144" member_type: VOTER last_known_addr { host: "127.19.52.66" port: 33431 } } peers { permanent_uuid: "01e9f5bd855b4f5b9abdc45a16316287" member_type: VOTER last_known_addr { host: "127.19.52.65" port: 36181 } } peers { permanent_uuid: "d6365d41e5504beba5648b3fe514eaee" member_type: VOTER last_known_addr { host: "127.19.52.67" port: 39047 } }
I20260812 06:55:56.705283 32028 raft_consensus.cc:374] T 90de5c43799e4ab288213e1c56a0e80a P a0155d1ee3c14b3cbdd171d2d606d144 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 06:55:56.705410 32028 raft_consensus.cc:729] T 90de5c43799e4ab288213e1c56a0e80a P a0155d1ee3c14b3cbdd171d2d606d144 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a0155d1ee3c14b3cbdd171d2d606d144, State: Initialized, Role: FOLLOWER
I20260812 06:55:56.705981 32028 consensus_queue.cc:260] T 90de5c43799e4ab288213e1c56a0e80a P a0155d1ee3c14b3cbdd171d2d606d144 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0155d1ee3c14b3cbdd171d2d606d144" member_type: VOTER last_known_addr { host: "127.19.52.66" port: 33431 } } peers { permanent_uuid: "01e9f5bd855b4f5b9abdc45a16316287" member_type: VOTER last_known_addr { host: "127.19.52.65" port: 36181 } } peers { permanent_uuid: "d6365d41e5504beba5648b3fe514eaee" member_type: VOTER last_known_addr { host: "127.19.52.67" port: 39047 } }
I20260812 06:55:56.706663 31859 heartbeater.cc:503] Master 127.19.52.126:42435 was elected leader, sending a full tablet report...
I20260812 06:55:56.707131 32028 ts_tablet_manager.cc:1428] T 90de5c43799e4ab288213e1c56a0e80a P a0155d1ee3c14b3cbdd171d2d606d144: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260812 06:55:56.744333 32034 raft_consensus.cc:482] T 90de5c43799e4ab288213e1c56a0e80a P a0155d1ee3c14b3cbdd171d2d606d144 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 06:55:56.744643 32034 raft_consensus.cc:504] T 90de5c43799e4ab288213e1c56a0e80a P a0155d1ee3c14b3cbdd171d2d606d144 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0155d1ee3c14b3cbdd171d2d606d144" member_type: VOTER last_known_addr { host: "127.19.52.66" port: 33431 } } peers { permanent_uuid: "01e9f5bd855b4f5b9abdc45a16316287" member_type: VOTER last_known_addr { host: "127.19.52.65" port: 36181 } } peers { permanent_uuid: "d6365d41e5504beba5648b3fe514eaee" member_type: VOTER last_known_addr { host: "127.19.52.67" port: 39047 } }
I20260812 06:55:56.746043 32034 leader_election.cc:288] T 90de5c43799e4ab288213e1c56a0e80a P a0155d1ee3c14b3cbdd171d2d606d144 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 01e9f5bd855b4f5b9abdc45a16316287 (127.19.52.65:36181), d6365d41e5504beba5648b3fe514eaee (127.19.52.67:39047)
I20260812 06:55:56.750080 31861 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:56.746327 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.67:39047 (local address 127.19.52.66:50515)
0812 06:55:56.746464 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0812 06:55:56.746482 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0812 06:55:56.746593 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:55:56.746873 (+ 280us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:55:56.746876 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:55:56.746898 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 06:55:56.747067 (+ 169us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:55:56.747076 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:55:56.748067 (+ 991us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:55:56.748072 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:55:56.748952 (+ 880us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:55:56.748961 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:55:56.749638 (+ 677us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:55:56.749642 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:55:56.749753 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:55:56.749771 (+ 18us) client_negotiation.cc:769] Sending connection context
0812 06:55:56.749860 (+ 89us) client_negotiation.cc:240] Negotiation successful
0812 06:55:56.749937 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":41,"mutex_wait_us":2}
I20260812 06:55:56.750083 32022 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:56.746436 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.52.66:50515 (local address 127.19.52.67:39047)
0812 06:55:56.746607 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0812 06:55:56.746614 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 06:55:56.746637 (+ 23us) server_negotiation.cc:407] Connection header received
0812 06:55:56.746700 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:55:56.746703 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:55:56.746759 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:55:56.746845 (+ 86us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 06:55:56.747192 (+ 347us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:56.747957 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:56.749090 (+ 1133us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:56.749504 (+ 414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:56.749646 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:55:56.749699 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0812 06:55:56.749879 (+ 180us) server_negotiation.cc:299] Negotiation successful
0812 06:55:56.749953 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260812 06:55:56.750367 32035 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:56.746327 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.65:36181 (local address 127.19.52.66:38627)
0812 06:55:56.746876 (+ 549us) negotiation.cc:107] Waiting for socket to connect
0812 06:55:56.746888 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 06:55:56.746974 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:55:56.747261 (+ 287us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:55:56.747264 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:55:56.747281 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 06:55:56.747391 (+ 110us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:55:56.747397 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:55:56.748325 (+ 928us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:55:56.748330 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:55:56.749172 (+ 842us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:55:56.749182 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:55:56.749921 (+ 739us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:55:56.749925 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:55:56.750031 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:55:56.750049 (+ 18us) client_negotiation.cc:769] Sending connection context
0812 06:55:56.750143 (+ 94us) client_negotiation.cc:240] Negotiation successful
0812 06:55:56.750219 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":464,"mutex_wait_us":139,"thread_start_us":155,"threads_started":1}
I20260812 06:55:56.750351 32023 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:56.746602 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.52.66:38627 (local address 127.19.52.65:36181)
0812 06:55:56.746750 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0812 06:55:56.746755 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 06:55:56.746966 (+ 211us) server_negotiation.cc:407] Connection header received
0812 06:55:56.747110 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:55:56.747113 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:55:56.747167 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:55:56.747246 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 06:55:56.747531 (+ 285us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:56.748203 (+ 672us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:56.749306 (+ 1103us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:56.749795 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:56.749925 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:55:56.749974 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0812 06:55:56.750163 (+ 189us) server_negotiation.cc:299] Negotiation successful
0812 06:55:56.750238 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260812 06:55:56.751010 31665 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "90de5c43799e4ab288213e1c56a0e80a" candidate_uuid: "a0155d1ee3c14b3cbdd171d2d606d144" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "01e9f5bd855b4f5b9abdc45a16316287" is_pre_election: true
I20260812 06:55:56.751205 31957 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "90de5c43799e4ab288213e1c56a0e80a" candidate_uuid: "a0155d1ee3c14b3cbdd171d2d606d144" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6365d41e5504beba5648b3fe514eaee" is_pre_election: true
I20260812 06:55:56.751328 31665 raft_consensus.cc:2454] T 90de5c43799e4ab288213e1c56a0e80a P 01e9f5bd855b4f5b9abdc45a16316287 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0155d1ee3c14b3cbdd171d2d606d144 in term 0.
I20260812 06:55:56.751524 31957 raft_consensus.cc:2454] T 90de5c43799e4ab288213e1c56a0e80a P d6365d41e5504beba5648b3fe514eaee [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0155d1ee3c14b3cbdd171d2d606d144 in term 0.
I20260812 06:55:56.752055 31740 leader_election.cc:302] T 90de5c43799e4ab288213e1c56a0e80a P a0155d1ee3c14b3cbdd171d2d606d144 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 01e9f5bd855b4f5b9abdc45a16316287, a0155d1ee3c14b3cbdd171d2d606d144, d6365d41e5504beba5648b3fe514eaee; no voters:
I20260812 06:55:56.752375 32034 raft_consensus.cc:2781] T 90de5c43799e4ab288213e1c56a0e80a P a0155d1ee3c14b3cbdd171d2d606d144 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 06:55:56.752472 32034 raft_consensus.cc:482] T 90de5c43799e4ab288213e1c56a0e80a P a0155d1ee3c14b3cbdd171d2d606d144 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 06:55:56.752524 32034 raft_consensus.cc:3037] T 90de5c43799e4ab288213e1c56a0e80a P a0155d1ee3c14b3cbdd171d2d606d144 [term 0 FOLLOWER]: Advancing to term 1
I20260812 06:55:56.753924 32034 raft_consensus.cc:504] T 90de5c43799e4ab288213e1c56a0e80a P a0155d1ee3c14b3cbdd171d2d606d144 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0155d1ee3c14b3cbdd171d2d606d144" member_type: VOTER last_known_addr { host: "127.19.52.66" port: 33431 } } peers { permanent_uuid: "01e9f5bd855b4f5b9abdc45a16316287" member_type: VOTER last_known_addr { host: "127.19.52.65" port: 36181 } } peers { permanent_uuid: "d6365d41e5504beba5648b3fe514eaee" member_type: VOTER last_known_addr { host: "127.19.52.67" port: 39047 } }
I20260812 06:55:56.754380 32034 leader_election.cc:288] T 90de5c43799e4ab288213e1c56a0e80a P a0155d1ee3c14b3cbdd171d2d606d144 [CANDIDATE]: Term 1 election: Requested vote from peers 01e9f5bd855b4f5b9abdc45a16316287 (127.19.52.65:36181), d6365d41e5504beba5648b3fe514eaee (127.19.52.67:39047)
I20260812 06:55:56.754804 31957 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "90de5c43799e4ab288213e1c56a0e80a" candidate_uuid: "a0155d1ee3c14b3cbdd171d2d606d144" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6365d41e5504beba5648b3fe514eaee"
I20260812 06:55:56.754879 31665 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "90de5c43799e4ab288213e1c56a0e80a" candidate_uuid: "a0155d1ee3c14b3cbdd171d2d606d144" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "01e9f5bd855b4f5b9abdc45a16316287"
I20260812 06:55:56.754978 31957 raft_consensus.cc:3037] T 90de5c43799e4ab288213e1c56a0e80a P d6365d41e5504beba5648b3fe514eaee [term 0 FOLLOWER]: Advancing to term 1
I20260812 06:55:56.755045 31665 raft_consensus.cc:3037] T 90de5c43799e4ab288213e1c56a0e80a P 01e9f5bd855b4f5b9abdc45a16316287 [term 0 FOLLOWER]: Advancing to term 1
I20260812 06:55:56.756155 31957 raft_consensus.cc:2454] T 90de5c43799e4ab288213e1c56a0e80a P d6365d41e5504beba5648b3fe514eaee [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a0155d1ee3c14b3cbdd171d2d606d144 in term 1.
I20260812 06:55:56.756292 31665 raft_consensus.cc:2454] T 90de5c43799e4ab288213e1c56a0e80a P 01e9f5bd855b4f5b9abdc45a16316287 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a0155d1ee3c14b3cbdd171d2d606d144 in term 1.
I20260812 06:55:56.756523 31738 leader_election.cc:302] T 90de5c43799e4ab288213e1c56a0e80a P a0155d1ee3c14b3cbdd171d2d606d144 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a0155d1ee3c14b3cbdd171d2d606d144, d6365d41e5504beba5648b3fe514eaee; no voters:
I20260812 06:55:56.756737 32034 raft_consensus.cc:2781] T 90de5c43799e4ab288213e1c56a0e80a P a0155d1ee3c14b3cbdd171d2d606d144 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 06:55:56.757030 32034 raft_consensus.cc:686] T 90de5c43799e4ab288213e1c56a0e80a P a0155d1ee3c14b3cbdd171d2d606d144 [term 1 LEADER]: Becoming Leader. State: Replica: a0155d1ee3c14b3cbdd171d2d606d144, State: Running, Role: LEADER
I20260812 06:55:56.757409 32034 consensus_queue.cc:237] T 90de5c43799e4ab288213e1c56a0e80a P a0155d1ee3c14b3cbdd171d2d606d144 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0155d1ee3c14b3cbdd171d2d606d144" member_type: VOTER last_known_addr { host: "127.19.52.66" port: 33431 } } peers { permanent_uuid: "01e9f5bd855b4f5b9abdc45a16316287" member_type: VOTER last_known_addr { host: "127.19.52.65" port: 36181 } } peers { permanent_uuid: "d6365d41e5504beba5648b3fe514eaee" member_type: VOTER last_known_addr { host: "127.19.52.67" port: 39047 } }
I20260812 06:55:56.760769 31505 catalog_manager.cc:5495] T 90de5c43799e4ab288213e1c56a0e80a P a0155d1ee3c14b3cbdd171d2d606d144 reported cstate change: term changed from 0 to 1, leader changed from <none> to a0155d1ee3c14b3cbdd171d2d606d144 (127.19.52.66). New cstate: current_term: 1 leader_uuid: "a0155d1ee3c14b3cbdd171d2d606d144" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0155d1ee3c14b3cbdd171d2d606d144" member_type: VOTER last_known_addr { host: "127.19.52.66" port: 33431 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "01e9f5bd855b4f5b9abdc45a16316287" member_type: VOTER last_known_addr { host: "127.19.52.65" port: 36181 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d6365d41e5504beba5648b3fe514eaee" member_type: VOTER last_known_addr { host: "127.19.52.67" port: 39047 } health_report { overall_health: UNKNOWN } } }
I20260812 06:55:56.776072 32020 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:56.772233 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43036 (local address 127.19.52.66:33431)
0812 06:55:56.772372 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0812 06:55:56.772377 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 06:55:56.772392 (+ 15us) server_negotiation.cc:407] Connection header received
0812 06:55:56.772605 (+ 213us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:55:56.772608 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:55:56.772682 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:55:56.772801 (+ 119us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 06:55:56.773073 (+ 272us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:56.773593 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:56.774515 (+ 922us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:56.774732 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:56.774844 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:55:56.775189 (+ 345us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 06:55:56.775632 (+ 443us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 06:55:56.775769 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0812 06:55:56.775838 (+ 69us) server_negotiation.cc:299] Negotiation successful
0812 06:55:56.775899 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260812 06:55:56.782388 31665 raft_consensus.cc:1261] T 90de5c43799e4ab288213e1c56a0e80a P 01e9f5bd855b4f5b9abdc45a16316287 [term 1 FOLLOWER]: Refusing update from remote peer a0155d1ee3c14b3cbdd171d2d606d144: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 06:55:56.782444 31957 raft_consensus.cc:1261] T 90de5c43799e4ab288213e1c56a0e80a P d6365d41e5504beba5648b3fe514eaee [term 1 FOLLOWER]: Refusing update from remote peer a0155d1ee3c14b3cbdd171d2d606d144: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 06:55:56.783203 32036 consensus_queue.cc:1035] T 90de5c43799e4ab288213e1c56a0e80a P a0155d1ee3c14b3cbdd171d2d606d144 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01e9f5bd855b4f5b9abdc45a16316287" member_type: VOTER last_known_addr { host: "127.19.52.65" port: 36181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 06:55:56.783412 32034 consensus_queue.cc:1035] T 90de5c43799e4ab288213e1c56a0e80a P a0155d1ee3c14b3cbdd171d2d606d144 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d6365d41e5504beba5648b3fe514eaee" member_type: VOTER last_known_addr { host: "127.19.52.67" port: 39047 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 06:55:56.793152 32041 mvcc.cc:205] Tried to move back new op lower bound from 7317576731771965440 to 7317576731681271808. Current Snapshot: MvccSnapshot[applied={T|T < 7317576731771965440}]
I20260812 06:55:56.801774 31505 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51516:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 06:55:56.802026 31505 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51516:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 06:55:56.804888 31505 catalog_manager.cc:5782] T 00000000000000000000000000000000 P dffab2aaa41a4948ba46209085f4ca00: Sending DeleteTablet for 3 replicas of tablet 90de5c43799e4ab288213e1c56a0e80a
I20260812 06:55:56.805881 31787 tablet_service.cc:1514] Processing DeleteTablet for tablet 90de5c43799e4ab288213e1c56a0e80a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 06:55:56 UTC) from {username='kudu', principal='kudu/127.19.52.126@KRBTEST.COM'} at 127.0.0.1:43022
I20260812 06:55:56.805896 31642 tablet_service.cc:1514] Processing DeleteTablet for tablet 90de5c43799e4ab288213e1c56a0e80a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 06:55:56 UTC) from {username='kudu', principal='kudu/127.19.52.126@KRBTEST.COM'} at 127.0.0.1:35288
I20260812 06:55:56.806470 32052 tablet_replica.cc:330] T 90de5c43799e4ab288213e1c56a0e80a P a0155d1ee3c14b3cbdd171d2d606d144: stopping tablet replica
I20260812 06:55:56.806536 32053 tablet_replica.cc:330] T 90de5c43799e4ab288213e1c56a0e80a P 01e9f5bd855b4f5b9abdc45a16316287: stopping tablet replica
I20260812 06:55:56.806694 31933 tablet_service.cc:1514] Processing DeleteTablet for tablet 90de5c43799e4ab288213e1c56a0e80a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 06:55:56 UTC) from {username='kudu', principal='kudu/127.19.52.126@KRBTEST.COM'} at 127.0.0.1:34422
I20260812 06:55:56.806879 32052 raft_consensus.cc:2229] T 90de5c43799e4ab288213e1c56a0e80a P a0155d1ee3c14b3cbdd171d2d606d144 [term 1 LEADER]: Raft consensus shutting down.
I20260812 06:55:56.806878 32053 raft_consensus.cc:2229] T 90de5c43799e4ab288213e1c56a0e80a P 01e9f5bd855b4f5b9abdc45a16316287 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 06:55:56.807235 32053 raft_consensus.cc:2258] T 90de5c43799e4ab288213e1c56a0e80a P 01e9f5bd855b4f5b9abdc45a16316287 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 06:55:56.807294 32052 raft_consensus.cc:2258] T 90de5c43799e4ab288213e1c56a0e80a P a0155d1ee3c14b3cbdd171d2d606d144 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 06:55:56.808656 19665 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu with pid 31572
I20260812 06:55:56.813225 32054 tablet_replica.cc:330] T 90de5c43799e4ab288213e1c56a0e80a P d6365d41e5504beba5648b3fe514eaee: stopping tablet replica
I20260812 06:55:56.813612 32054 raft_consensus.cc:2229] T 90de5c43799e4ab288213e1c56a0e80a P d6365d41e5504beba5648b3fe514eaee [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 06:55:56.813638 32052 ts_tablet_manager.cc:1905] T 90de5c43799e4ab288213e1c56a0e80a P a0155d1ee3c14b3cbdd171d2d606d144: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 06:55:56.813992 32054 raft_consensus.cc:2258] T 90de5c43799e4ab288213e1c56a0e80a P d6365d41e5504beba5648b3fe514eaee [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 06:55:56.818338 32054 ts_tablet_manager.cc:1905] T 90de5c43799e4ab288213e1c56a0e80a P d6365d41e5504beba5648b3fe514eaee: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260812 06:55:56.818547 31489 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.19.52.65:36181 (error 108)
I20260812 06:55:56.819542 19665 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu with pid 31718
I20260812 06:55:56.819820 32052 ts_tablet_manager.cc:1918] T 90de5c43799e4ab288213e1c56a0e80a P a0155d1ee3c14b3cbdd171d2d606d144: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 06:55:56.819931 32052 log.cc:1196] T 90de5c43799e4ab288213e1c56a0e80a P a0155d1ee3c14b3cbdd171d2d606d144: Deleting WAL directory at /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786517687384214-19665-0/minicluster-data/ts-1/wal/wals/90de5c43799e4ab288213e1c56a0e80a
I20260812 06:55:56.821377 32054 ts_tablet_manager.cc:1918] T 90de5c43799e4ab288213e1c56a0e80a P d6365d41e5504beba5648b3fe514eaee: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 06:55:56.821503 32054 log.cc:1196] T 90de5c43799e4ab288213e1c56a0e80a P d6365d41e5504beba5648b3fe514eaee: Deleting WAL directory at /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786517687384214-19665-0/minicluster-data/ts-2/wal/wals/90de5c43799e4ab288213e1c56a0e80a
I20260812 06:55:56.821892 32054 ts_tablet_manager.cc:1939] T 90de5c43799e4ab288213e1c56a0e80a P d6365d41e5504beba5648b3fe514eaee: Deleting consensus metadata
I20260812 06:55:56.823494 31486 catalog_manager.cc:4841] TS d6365d41e5504beba5648b3fe514eaee (127.19.52.67:39047): tablet 90de5c43799e4ab288213e1c56a0e80a (table test-table [id=80205dfabf2a47b3b6344e1c7c02e583]) successfully deleted
I20260812 06:55:56.823828 32019 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:56.823268 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.65:36181 (local address 127.0.0.1:35300)
0812 06:55:56.823622 (+ 354us) negotiation.cc:107] Waiting for socket to connect
0812 06:55:56.823702 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.52.65:36181: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":258}
W20260812 06:55:56.824020 31489 catalog_manager.cc:4584] TS 01e9f5bd855b4f5b9abdc45a16316287 (127.19.52.65:36181): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 90de5c43799e4ab288213e1c56a0e80a: Network error: Client connection negotiation failed: client connection to 127.19.52.65:36181: connect: Connection refused (error 111)
I20260812 06:55:56.828804 19665 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu with pid 31864
I20260812 06:55:56.829381 32019 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:56.828970 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.66:33431 (local address 127.0.0.1:43050)
0812 06:55:56.829208 (+ 238us) negotiation.cc:107] Waiting for socket to connect
0812 06:55:56.829270 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.52.66:33431: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":143}
I20260812 06:55:56.838828 19665 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu with pid 31466
2026-08-12T06:55:56Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationSecureCluster (2537 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegration
Loading random data
Initializing database '/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786517687384214-19665-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 06:55:56 dist-test-slave-65mx krb5kdc[32059](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 06:55:56 dist-test-slave-65mx krb5kdc[32059](info): set up 2 sockets
Aug 12 06:55:56 dist-test-slave-65mx krb5kdc[32059](info): commencing operation
krb5kdc: starting...
W20260812 06:55:57.977766 19665 mini_kdc.cc:121] Time spent starting KDC: real 1.112s user 0.002s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786517687384214-19665-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786517687384214-19665-0/krb5kdc/test-user.keytab.
Aug 12 06:55:58 dist-test-slave-65mx krb5kdc[32059](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517758, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T06:55:58Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T06:55:58Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.19.52.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.19.52.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.19.52.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786517687384214-19665-0/krb5kdc/rangeradmin_127.19.52.84@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.19.52.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786517687384214-19665-0/krb5kdc/rangeradmin_127.19.52.84@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.19.52.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.19.52.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.19.52.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786517687384214-19665-0/krb5kdc/rangerlookup_127.19.52.84@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.19.52.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786517687384214-19665-0/krb5kdc/rangerlookup_127.19.52.84@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.19.52.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.19.52.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.19.52.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786517687384214-19665-0/krb5kdc/HTTP_127.19.52.84@KRBTEST.COM.keytab.
Entry for principal HTTP/127.19.52.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786517687384214-19665-0/krb5kdc/HTTP_127.19.52.84@KRBTEST.COM.keytab.
I20260812 06:55:58.185329 19665 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786517687384214-19665-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskG7Vh1a/build/debug/bin/postgres/pg_ctl -D /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786517687384214-19665-0/minicluster-data/postgres -l logfile start
2026-08-12 06:56:01.050 UTC [32091] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-12 06:56:01.050 UTC [32091] LOG: listening on IPv4 address "127.19.52.84", port 59323
2026-08-12 06:56:01.054 UTC [32091] LOG: listening on Unix socket "/tmp/.s.PGSQL.59323"
2026-08-12 06:56:01.094 UTC [32094] LOG: database system was shut down at 2026-08-12 06:55:59 UTC
2026-08-12 06:56:01.099 UTC [32091] LOG: database system is ready to accept connections
I20260812 06:56:02.131806 19665 mini_postgres.cc:96] Postgres bound to 59323
2026-08-12 06:56:02.139 UTC [32102] FATAL: database "slave" does not exist
127.19.52.84:59323 - accepting connections
I20260812 06:56:02.140823 19665 mini_ranger.cc:161] Starting Ranger...
I20260812 06:56:02.158581 19665 mini_ranger.cc:84] Created miniranger Postgres user
I20260812 06:56:02.775877 19665 mini_ranger.cc:87] Created ranger Postgres database
I20260812 06:56:02.775980 19665 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786517687384214-19665-0/minicluster-data/ranger-admin
2026-08-12 06:56:03,078 [I] DB FLAVOR :POSTGRES
2026-08-12 06:56:03,079 [I] --------- Verifying Ranger DB connection ---------
2026-08-12 06:56:03,079 [I] Checking connection..
2026-08-12 06:56:03,079 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskG7Vh1a/build/debug/bin/postgresql.jar:/tmp/dist-test-taskG7Vh1a/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.19.52.84:59323/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-12 06:56:03,347 [I] Checking connection passed.
2026-08-12 06:56:03,347 [I] --------- Verifying version history table ---------
2026-08-12 06:56:03,347 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskG7Vh1a/build/debug/bin/postgresql.jar:/tmp/dist-test-taskG7Vh1a/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.19.52.84:59323/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-12 06:56:03,604 [I] Table x_db_version_h does not exist in database ranger
2026-08-12 06:56:03,604 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskG7Vh1a/build/debug/bin/postgresql.jar:/tmp/dist-test-taskG7Vh1a/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.19.52.84:59323/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-12 06:56:03,862 [I] Table x_db_version_h does not exist in database ranger
2026-08-12 06:56:03,862 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-12 06:56:03,862 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskG7Vh1a/build/debug/bin/postgresql.jar:/tmp/dist-test-taskG7Vh1a/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.19.52.84:59323/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskG7Vh1a/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-12 06:56:04.132 UTC [32171] WARNING: there is no transaction in progress
2026-08-12 06:56:04,139 [I] create_dbversion_catalog.sql file imported successfully
2026-08-12 06:56:04,140 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskG7Vh1a/build/debug/bin/postgresql.jar:/tmp/dist-test-taskG7Vh1a/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.19.52.84:59323/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-12 06:56:04,427 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-12 06:56:04,427 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-12 06:56:04,427 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskG7Vh1a/build/debug/bin/postgresql.jar:/tmp/dist-test-taskG7Vh1a/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.19.52.84:59323/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-12 06:56:04,697 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskG7Vh1a/build/debug/bin/postgresql.jar:/tmp/dist-test-taskG7Vh1a/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.19.52.84:59323/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-12 06:56:04,968 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskG7Vh1a/build/debug/bin/postgresql.jar:/tmp/dist-test-taskG7Vh1a/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.19.52.84:59323/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-65mx.c.gcp-upstream.internal','N') ;"
2026-08-12 06:56:05,226 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskG7Vh1a/build/debug/bin/postgresql.jar:/tmp/dist-test-taskG7Vh1a/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.19.52.84:59323/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-12 06:56:05,498 [I] Table x_portal_user does not exist in database ranger
2026-08-12 06:56:05,498 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskG7Vh1a/build/debug/bin/postgresql.jar:/tmp/dist-test-taskG7Vh1a/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.19.52.84:59323/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-12 06:56:05,755 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-12 06:56:05,755 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskG7Vh1a/build/debug/bin/postgresql.jar:/tmp/dist-test-taskG7Vh1a/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.19.52.84:59323/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-12 06:56:06,035 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskG7Vh1a/build/debug/bin/postgresql.jar:/tmp/dist-test-taskG7Vh1a/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.19.52.84:59323/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-12 06:56:06,308 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-12 06:56:06,309 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskG7Vh1a/build/debug/bin/postgresql.jar:/tmp/dist-test-taskG7Vh1a/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.19.52.84:59323/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskG7Vh1a/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-12 06:56:07.401 UTC [32292] WARNING: there is no transaction in progress
2026-08-12 06:56:07.421 UTC [32292] WARNING: there is no transaction in progress
2026-08-12 06:56:07.441 UTC [32292] WARNING: there is no transaction in progress
2026-08-12 06:56:07.460 UTC [32292] WARNING: there is no transaction in progress
2026-08-12 06:56:07.481 UTC [32292] WARNING: there is no transaction in progress
2026-08-12 06:56:07.501 UTC [32292] WARNING: there is no transaction in progress
2026-08-12 06:56:07.567 UTC [32292] WARNING: there is no transaction in progress
2026-08-12 06:56:07.576 UTC [32292] WARNING: there is no transaction in progress
2026-08-12 06:56:07.594 UTC [32292] WARNING: there is no transaction in progress
2026-08-12 06:56:07.607 UTC [32292] WARNING: there is no transaction in progress
2026-08-12 06:56:07.619 UTC [32292] WARNING: there is no transaction in progress
2026-08-12 06:56:07.638 UTC [32292] WARNING: there is no transaction in progress
2026-08-12 06:56:07.650 UTC [32292] WARNING: there is no transaction in progress
2026-08-12 06:56:07.659 UTC [32292] WARNING: there is no transaction in progress
2026-08-12 06:56:07.669 UTC [32292] WARNING: there is no transaction in progress
2026-08-12 06:56:07.677 UTC [32292] WARNING: there is no transaction in progress
2026-08-12 06:56:08,491 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-12 06:56:08,492 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskG7Vh1a/build/debug/bin/postgresql.jar:/tmp/dist-test-taskG7Vh1a/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.19.52.84:59323/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-12 06:56:08,784 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-12 06:56:08,785 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskG7Vh1a/build/debug/bin/postgresql.jar:/tmp/dist-test-taskG7Vh1a/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.19.52.84:59323/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-12 06:56:09,052 [I] Table x_portal_user already exists in database 'ranger'
2026-08-12 06:56:09,053 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskG7Vh1a/build/debug/bin/postgresql.jar:/tmp/dist-test-taskG7Vh1a/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.19.52.84:59323/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-12 06:56:09,303 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-12 06:56:09,304 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskG7Vh1a/build/debug/bin/postgresql.jar:/tmp/dist-test-taskG7Vh1a/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.19.52.84:59323/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-12 06:56:09,583 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskG7Vh1a/build/debug/bin/postgresql.jar:/tmp/dist-test-taskG7Vh1a/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.19.52.84:59323/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-12 06:56:09,836 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskG7Vh1a/build/debug/bin/postgresql.jar:/tmp/dist-test-taskG7Vh1a/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.19.52.84:59323/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-65mx.c.gcp-upstream.internal';"
2026-08-12 06:56:10,098 [I] CORE_DB_SCHEMA import status has been updated
2026-08-12 06:56:10,099 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskG7Vh1a/build/debug/bin/postgresql.jar:/tmp/dist-test-taskG7Vh1a/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.19.52.84:59323/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-12 06:56:10,381 [I] DB_PATCHES have already been applied
I20260812 06:56:10.386972 19665 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786517687384214-19665-0/minicluster-data/ranger-admin:/tmp/dist-test-taskG7Vh1a/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskG7Vh1a/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskG7Vh1a/build/debug/bin/postgresql.jar:/tmp/dist-test-taskG7Vh1a/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260812 06:56:10.387054 19665 mini_ranger.cc:193] Using host: 127.19.52.84
I20260812 06:56:10.391799 19665 mini_ranger.cc:227] Ranger admin URL: http://127.19.52.84:59703
Aug 12, 2026 6:56:10 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 12, 2026 6:56:10 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 12, 2026 6:56:10 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786517687384214-19665-0/minicluster-data/ranger-admin/ews/webapp
Aug 12, 2026 6:56:10 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786517687384214-19665-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 12, 2026 6:56:10 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786517687384214-19665-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 12, 2026 6:56:11 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786517687384214-19665-0/minicluster-data/ranger-admin/ews/webapp].
Aug 12, 2026 6:56:11 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260812 06:56:11.627648 19665 mini_ranger.cc:160] Time spent starting Ranger: real 9.487s user 0.002s sys 0.012s
Aug 12, 2026 6:56:33 AM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 12, 2026 6:56:33 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.PublicAPIsv2
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.UserREST
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.MetricsREST
Aug 12, 2026 6:56:33 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 12, 2026 6:56:33 AM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 12, 2026 6:56:33 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 12, 2026 6:56:33 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 12, 2026 6:56:33 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 12, 2026 6:56:33 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 12, 2026 6:56:33 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 12, 2026 6:56:33 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 12, 2026 6:56:33 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 12, 2026 6:56:33 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 12, 2026 6:56:33 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 12, 2026 6:56:33 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 12, 2026 6:56:33 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 12, 2026 6:56:33 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 12, 2026 6:56:33 AM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 12, 2026 6:56:34 AM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260812 06:56:35.206615 19665 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.19.52.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.19.52.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.19.52.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786517687384214-19665-0/krb5kdc/rangerkms_127.19.52.84@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.19.52.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786517687384214-19665-0/krb5kdc/rangerkms_127.19.52.84@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.19.52.84@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.19.52.84@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.19.52.84@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786517687384214-19665-0/krb5kdc/HTTP_127.19.52.84@KRBTEST.COM.keytab.
Entry for principal HTTP/127.19.52.84@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786517687384214-19665-0/krb5kdc/HTTP_127.19.52.84@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 12 06:56:35 dist-test-slave-65mx krb5kdc[32059](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517795, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260812 06:56:35.308722 19665 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260812 06:56:35.320392 19665 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260812 06:56:36.286721 19665 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260812 06:56:36.286826 19665 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786517687384214-19665-0/minicluster-data/ranger-kms
I20260812 06:56:36.286934 19665 mini_ranger_kms.cc:227] Using postgres at 127.19.52.84:59323
2026-08-12 06:56:36,564 [I] DB FLAVOR :POSTGRES
2026-08-12 06:56:36,565 [I] --------- Verifying Ranger DB connection ---------
2026-08-12 06:56:36,565 [I] Checking connection
2026-08-12 06:56:36,565 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskG7Vh1a/build/debug/bin/postgresql.jar:/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786517687384214-19665-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.19.52.84:59323/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-12 06:56:36,818 [I] connection success
2026-08-12 06:56:36,818 [I] --------- Verifying Ranger DB tables ---------
2026-08-12 06:56:36,819 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-12 06:56:36,819 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskG7Vh1a/build/debug/bin/postgresql.jar:/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786517687384214-19665-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.19.52.84:59323/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-12 06:56:37,085 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-12 06:56:37,085 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-12 06:56:37,085 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-12 06:56:37,085 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskG7Vh1a/build/debug/bin/postgresql.jar:/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786517687384214-19665-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.19.52.84:59323/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786517687384214-19665-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-12 06:56:37,383 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260812 06:56:37.925760 19665 mini_ranger_kms.cc:310] Created kms service
I20260812 06:56:38.111670 19665 mini_ranger_kms.cc:326] Created kudu user
I20260812 06:56:38.159968 19665 mini_ranger_kms.cc:343] Created rangerkms user
Aug 12 06:56:38 dist-test-slave-65mx krb5kdc[32059](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517795, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.19.52.84@KRBTEST.COM
I20260812 06:56:38.599802 19665 mini_ranger_kms.cc:384] Added ranger policy
I20260812 06:56:38.599975 19665 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786517687384214-19665-0/minicluster-data/ranger-kms:/tmp/dist-test-taskG7Vh1a/build/debug/bin/postgresql.jar:/tmp/dist-test-taskG7Vh1a/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskG7Vh1a/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260812 06:56:38.600001 19665 mini_ranger_kms.cc:240] Using host: 127.19.52.84
I20260812 06:56:38.605592 19665 mini_ranger_kms.cc:276] Ranger KMS PID: 32489
I20260812 06:56:38.605762 19665 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.19.52.84:60953
Aug 12, 2026 6:56:39 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 12, 2026 6:56:39 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 12, 2026 6:56:39 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786517687384214-19665-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 12, 2026 6:56:39 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786517687384214-19665-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 12, 2026 6:56:40 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786517687384214-19665-0/minicluster-data/ranger-kms/ews/webapp].
Aug 12, 2026 6:56:40 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
I20260812 06:56:41.062237 19665 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 5.754s user 0.000s sys 0.261s
I20260812 06:56:41.062377 19665 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260812 06:56:41.062498 19665 mini_ranger_kms.cc:401] 127.19.52.84:60953/kms/v1/keys
Aug 12 06:56:41 dist-test-slave-65mx krb5kdc[32059](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786517801, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.19.52.84@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Aug 12 06:56:41 dist-test-slave-65mx krb5kdc[32059](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517801, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.19.52.84@KRBTEST.COM for HTTP/127.19.52.84@KRBTEST.COM
Aug 12 06:56:41 dist-test-slave-65mx krb5kdc[32059](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517801, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.19.52.84@KRBTEST.COM for HTTP/127.19.52.84@KRBTEST.COM
Aug 12 06:56:44 dist-test-slave-65mx krb5kdc[32059](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517804, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.19.52.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.52.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.52.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786517687384214-19665-0/krb5kdc/kudu_127.19.52.126.keytab.
Entry for principal kudu/127.19.52.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786517687384214-19665-0/krb5kdc/kudu_127.19.52.126.keytab.
I20260812 06:56:44.553956 19665 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu
/tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786517687384214-19665-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786517687384214-19665-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786517687384214-19665-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786517687384214-19665-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.52.126:37035
--webserver_interface=127.19.52.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786517687384214-19665-0/krb5kdc/kudu_127.19.52.126.keytab
--principal=kudu/127.19.52.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.52.84:40013
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.19.52.126:37035
--ranger_config_path=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786517687384214-19665-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.19.52.84:60953/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786517687384214-19665-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786517687384214-19665-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786517687384214-19665-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 06:56:44.701859 32549 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:56:44.702160 32549 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:56:44.702234 32549 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 06:56:44.706648 32549 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 06:56:44.706733 32549 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 06:56:44.706753 32549 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 06:56:44.706771 32549 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 06:56:44.706787 32549 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 06:56:44.712883 32549 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.52.84:40013
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786517687384214-19665-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786517687384214-19665-0/minicluster-data/master-0/wal
--ranger_kms_url=127.19.52.84:60953/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.19.52.126:37035
--ranger_config_path=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786517687384214-19665-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786517687384214-19665-0/krb5kdc/kudu_127.19.52.126.keytab
--principal=kudu/127.19.52.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.52.126:37035
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786517687384214-19665-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.19.52.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32549
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786517687384214-19665-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 06:15:54 UTC on ccycloud.quasar-rnfzce.root.comops.site
build id 4581989
I20260812 06:56:44.714092 32549 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:56:44.715193 32549 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 06:56:44.726084 32554 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:56:44.726027 32557 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:56:44.726308 32549 server_base.cc:1034] running on GCE node
W20260812 06:56:44.726328 32555 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:56:44.727303 32549 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:56:44.728533 32549 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:56:44.729801 32549 hybrid_clock.cc:648] HybridClock initialized: now 1786517804729782 us; error 42 us; skew 500 ppm
Aug 12 06:56:44 dist-test-slave-65mx krb5kdc[32059](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517804, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.52.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 06:56:44.733348 32549 init.cc:377] Logged in from keytab as kudu/127.19.52.126@KRBTEST.COM (short username kudu)
I20260812 06:56:44.734591 32549 webserver.cc:460] Webserver started at http://127.19.52.126:33811/ using document root <none> and password file <none>
I20260812 06:56:44.735196 32549 fs_manager.cc:359] Metadata directory not provided
I20260812 06:56:44.735282 32549 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:56:44.735529 32549 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 12 06:56:44 dist-test-slave-65mx krb5kdc[32059](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517804, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.52.126@KRBTEST.COM for HTTP/127.19.52.84@KRBTEST.COM
I20260812 06:56:44.788940 32549 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786517687384214-19665-0/minicluster-data/master-0/data/instance:
uuid: "9c57f5da932a430c874acc23a69c8fbe"
format_stamp: "Formatted at 2026-08-12 06:56:44 on dist-test-slave-65mx"
server_key: "4d2332b52df7a654bdb6fd07f76ae2e9"
server_key_iv: "179ea693baa3c8eef360049880696244"
server_key_version: "kuduclusterkey@0"
I20260812 06:56:44.789614 32549 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786517687384214-19665-0/minicluster-data/master-0/wal/instance:
uuid: "9c57f5da932a430c874acc23a69c8fbe"
format_stamp: "Formatted at 2026-08-12 06:56:44 on dist-test-slave-65mx"
server_key: "4d2332b52df7a654bdb6fd07f76ae2e9"
server_key_iv: "179ea693baa3c8eef360049880696244"
server_key_version: "kuduclusterkey@0"
I20260812 06:56:44.793834 32549 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.001s
I20260812 06:56:44.809994 32564 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 06:56:44.811497 32549 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260812 06:56:44.811671 32549 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786517687384214-19665-0/minicluster-data/master-0/data,/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786517687384214-19665-0/minicluster-data/master-0/wal
uuid: "9c57f5da932a430c874acc23a69c8fbe"
format_stamp: "Formatted at 2026-08-12 06:56:44 on dist-test-slave-65mx"
server_key: "4d2332b52df7a654bdb6fd07f76ae2e9"
server_key_iv: "179ea693baa3c8eef360049880696244"
server_key_version: "kuduclusterkey@0"
I20260812 06:56:44.811828 32549 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786517687384214-19665-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786517687384214-19665-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786517687384214-19665-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:56:44.836177 32549 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 06:56:44.839501 32549 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 06:56:44.839732 32549 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:56:44.848232 32549 rpc_server.cc:307] RPC server started. Bound to: 127.19.52.126:37035
I20260812 06:56:44.848310 32619 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.52.126:37035 every 8 connection(s)
I20260812 06:56:44.849474 32549 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786517687384214-19665-0/minicluster-data/master-0/data/info.pb
I20260812 06:56:44.852604 32621 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 06:56:44.855561 19665 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu as pid 32549
I20260812 06:56:44.855717 19665 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786517687384214-19665-0/minicluster-data/master-0/wal/instance
I20260812 06:56:44.856035 19665 external_mini_cluster.cc:1428] Setting key 6709189f07dd8c7e979cd72ddd40c8c3
I20260812 06:56:44.859364 32621 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9c57f5da932a430c874acc23a69c8fbe: Bootstrap starting.
I20260812 06:56:44.862291 32621 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9c57f5da932a430c874acc23a69c8fbe: Neither blocks nor log segments found. Creating new log.
I20260812 06:56:44.863181 32621 log.cc:824] T 00000000000000000000000000000000 P 9c57f5da932a430c874acc23a69c8fbe: Log is configured to *not* fsync() on all Append() calls
I20260812 06:56:44.865677 32621 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9c57f5da932a430c874acc23a69c8fbe: No bootstrap required, opened a new log
Aug 12 06:56:44 dist-test-slave-65mx krb5kdc[32059](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517804, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.19.52.126@KRBTEST.COM
I20260812 06:56:44.870052 32621 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9c57f5da932a430c874acc23a69c8fbe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9c57f5da932a430c874acc23a69c8fbe" member_type: VOTER last_known_addr { host: "127.19.52.126" port: 37035 } }
I20260812 06:56:44.870333 32621 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9c57f5da932a430c874acc23a69c8fbe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 06:56:44.870446 32621 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9c57f5da932a430c874acc23a69c8fbe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9c57f5da932a430c874acc23a69c8fbe, State: Initialized, Role: FOLLOWER
I20260812 06:56:44.870931 32621 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9c57f5da932a430c874acc23a69c8fbe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9c57f5da932a430c874acc23a69c8fbe" member_type: VOTER last_known_addr { host: "127.19.52.126" port: 37035 } }
I20260812 06:56:44.871102 32621 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9c57f5da932a430c874acc23a69c8fbe [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 06:56:44.871191 32621 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9c57f5da932a430c874acc23a69c8fbe [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 06:56:44.871338 32621 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9c57f5da932a430c874acc23a69c8fbe [term 0 FOLLOWER]: Advancing to term 1
I20260812 06:56:44.872524 32621 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9c57f5da932a430c874acc23a69c8fbe [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9c57f5da932a430c874acc23a69c8fbe" member_type: VOTER last_known_addr { host: "127.19.52.126" port: 37035 } }
I20260812 06:56:44.872921 32624 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:56:44.857800 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55394 (local address 127.19.52.126:37035)
0812 06:56:44.858362 (+ 562us) server_negotiation.cc:206] Beginning negotiation
0812 06:56:44.858383 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0812 06:56:44.858429 (+ 46us) server_negotiation.cc:407] Connection header received
0812 06:56:44.859252 (+ 823us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:56:44.859275 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:56:44.859733 (+ 458us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:56:44.860111 (+ 378us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:56:44.861213 (+ 1102us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:44.862733 (+ 1520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:44.863998 (+ 1265us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:44.864507 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:44.864730 (+ 223us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:56:44.867736 (+ 3006us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:56:44.867742 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:56:44.867755 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:56:44.867784 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:56:44.869635 (+ 1851us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:56:44.870172 (+ 537us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:56:44.870176 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:56:44.870181 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:56:44.870288 (+ 107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:56:44.870630 (+ 342us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:56:44.870634 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:56:44.870644 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:56:44.871154 (+ 510us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:56:44.871354 (+ 200us) server_negotiation.cc:1035] Waiting for connection context
0812 06:56:44.871680 (+ 326us) server_negotiation.cc:299] Negotiation successful
0812 06:56:44.871964 (+ 284us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":382,"thread_start_us":179,"threads_started":1}
I20260812 06:56:44.872931 32621 leader_election.cc:302] T 00000000000000000000000000000000 P 9c57f5da932a430c874acc23a69c8fbe [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9c57f5da932a430c874acc23a69c8fbe; no voters:
I20260812 06:56:44.873333 32621 leader_election.cc:288] T 00000000000000000000000000000000 P 9c57f5da932a430c874acc23a69c8fbe [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 06:56:44.873454 32626 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9c57f5da932a430c874acc23a69c8fbe [term 1 FOLLOWER]: Leader election won for term 1
I20260812 06:56:44.873674 32626 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9c57f5da932a430c874acc23a69c8fbe [term 1 LEADER]: Becoming Leader. State: Replica: 9c57f5da932a430c874acc23a69c8fbe, State: Running, Role: LEADER
I20260812 06:56:44.874032 32626 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9c57f5da932a430c874acc23a69c8fbe [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9c57f5da932a430c874acc23a69c8fbe" member_type: VOTER last_known_addr { host: "127.19.52.126" port: 37035 } }
I20260812 06:56:44.874640 32621 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9c57f5da932a430c874acc23a69c8fbe [sys.catalog]: configured and running, proceeding with master startup.
I20260812 06:56:44.875860 32626 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9c57f5da932a430c874acc23a69c8fbe [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9c57f5da932a430c874acc23a69c8fbe. Latest consensus state: current_term: 1 leader_uuid: "9c57f5da932a430c874acc23a69c8fbe" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9c57f5da932a430c874acc23a69c8fbe" member_type: VOTER last_known_addr { host: "127.19.52.126" port: 37035 } } }
I20260812 06:56:44.875988 32626 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9c57f5da932a430c874acc23a69c8fbe [sys.catalog]: This master's current role is: LEADER
I20260812 06:56:44.876334 32627 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9c57f5da932a430c874acc23a69c8fbe [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9c57f5da932a430c874acc23a69c8fbe" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9c57f5da932a430c874acc23a69c8fbe" member_type: VOTER last_known_addr { host: "127.19.52.126" port: 37035 } } }
I20260812 06:56:44.876368 32636 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 06:56:44.876466 32627 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9c57f5da932a430c874acc23a69c8fbe [sys.catalog]: This master's current role is: LEADER
I20260812 06:56:44.880125 32636 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 06:56:44.880681 32621 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786517687384214-19665-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260812 06:56:44.891935 32636 catalog_manager.cc:1347] Generated new cluster ID: 4c029cb10a7c49609e17d0b269a5d392
I20260812 06:56:44.892043 32636 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 06:56:44.923305 32636 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 06:56:44.924343 32636 catalog_manager.cc:1504] Loading token signing keys...
I20260812 06:56:44.931242 32636 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9c57f5da932a430c874acc23a69c8fbe: Generated new TSK 0
I20260812 06:56:44.931968 32636 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.19.52.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.52.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.52.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786517687384214-19665-0/krb5kdc/kudu_127.19.52.65.keytab.
Entry for principal kudu/127.19.52.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786517687384214-19665-0/krb5kdc/kudu_127.19.52.65.keytab.
I20260812 06:56:45.701853 19665 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu
/tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786517687384214-19665-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786517687384214-19665-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786517687384214-19665-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786517687384214-19665-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.19.52.84:60953/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.52.65:0
--local_ip_for_outbound_sockets=127.19.52.65
--webserver_interface=127.19.52.65
--webserver_port=0
--tserver_master_addrs=127.19.52.126:37035
--keytab_file=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786517687384214-19665-0/krb5kdc/kudu_127.19.52.65.keytab
--principal=kudu/127.19.52.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.52.84:40013
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786517687384214-19665-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786517687384214-19665-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786517687384214-19665-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 06:56:45.872910 32670 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:56:45.873234 32670 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:56:45.873354 32670 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 06:56:45.878149 32670 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 06:56:45.878243 32670 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 06:56:45.878346 32670 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.52.65
I20260812 06:56:45.884747 32670 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.52.84:40013
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786517687384214-19665-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786517687384214-19665-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.19.52.84:60953/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786517687384214-19665-0/krb5kdc/kudu_127.19.52.65.keytab
--principal=kudu/127.19.52.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.52.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786517687384214-19665-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.19.52.65
--webserver_port=0
--tserver_master_addrs=127.19.52.126:37035
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32670
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.52.65
--log_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786517687384214-19665-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 06:15:54 UTC on ccycloud.quasar-rnfzce.root.comops.site
build id 4581989
I20260812 06:56:45.886337 32670 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:56:45.887758 32670 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 06:56:45.900727 32680 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:56:45.900739 32679 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:56:45.900671 32682 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:56:45.901660 32670 server_base.cc:1034] running on GCE node
I20260812 06:56:45.902252 32670 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:56:45.903182 32670 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:56:45.904410 32670 hybrid_clock.cc:648] HybridClock initialized: now 1786517805904411 us; error 59 us; skew 500 ppm
Aug 12 06:56:45 dist-test-slave-65mx krb5kdc[32059](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517805, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.52.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 06:56:45.908939 32670 init.cc:377] Logged in from keytab as kudu/127.19.52.65@KRBTEST.COM (short username kudu)
I20260812 06:56:45.910197 32670 webserver.cc:460] Webserver started at http://127.19.52.65:42363/ using document root <none> and password file <none>
I20260812 06:56:45.910821 32670 fs_manager.cc:359] Metadata directory not provided
I20260812 06:56:45.910923 32670 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:56:45.911155 32670 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 12 06:56:45 dist-test-slave-65mx krb5kdc[32059](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517805, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.52.65@KRBTEST.COM for HTTP/127.19.52.84@KRBTEST.COM
I20260812 06:56:45.926390 32670 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786517687384214-19665-0/minicluster-data/ts-0/data/instance:
uuid: "c0dd0a83e72848edb10b2662b59172b3"
format_stamp: "Formatted at 2026-08-12 06:56:45 on dist-test-slave-65mx"
server_key: "f4ee23acd7f6b068c9d6c3c532bf5397"
server_key_iv: "bbbbd547f0fc864adf1ef3fe3d4d0af4"
server_key_version: "kuduclusterkey@0"
I20260812 06:56:45.927181 32670 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786517687384214-19665-0/minicluster-data/ts-0/wal/instance:
uuid: "c0dd0a83e72848edb10b2662b59172b3"
format_stamp: "Formatted at 2026-08-12 06:56:45 on dist-test-slave-65mx"
server_key: "f4ee23acd7f6b068c9d6c3c532bf5397"
server_key_iv: "bbbbd547f0fc864adf1ef3fe3d4d0af4"
server_key_version: "kuduclusterkey@0"
I20260812 06:56:45.933159 32670 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.001s
I20260812 06:56:45.958691 32689 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 06:56:45.960706 32670 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.000s
I20260812 06:56:45.960880 32670 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786517687384214-19665-0/minicluster-data/ts-0/data,/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786517687384214-19665-0/minicluster-data/ts-0/wal
uuid: "c0dd0a83e72848edb10b2662b59172b3"
format_stamp: "Formatted at 2026-08-12 06:56:45 on dist-test-slave-65mx"
server_key: "f4ee23acd7f6b068c9d6c3c532bf5397"
server_key_iv: "bbbbd547f0fc864adf1ef3fe3d4d0af4"
server_key_version: "kuduclusterkey@0"
I20260812 06:56:45.961040 32670 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786517687384214-19665-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786517687384214-19665-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786517687384214-19665-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:56:45.992990 32670 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 06:56:45.996992 32670 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 06:56:45.997207 32670 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:56:45.998071 32670 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 06:56:45.999140 32670 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 06:56:45.999212 32670 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 06:56:45.999277 32670 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 06:56:45.999305 32670 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 06:56:46.011925 32670 rpc_server.cc:307] RPC server started. Bound to: 127.19.52.65:43711
I20260812 06:56:46.013203 32670 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786517687384214-19665-0/minicluster-data/ts-0/data/info.pb
I20260812 06:56:46.018297 19665 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu as pid 32670
I20260812 06:56:46.018452 19665 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786517687384214-19665-0/minicluster-data/ts-0/wal/instance
I20260812 06:56:46.018790 19665 external_mini_cluster.cc:1428] Setting key dec40986fddc9a42e3fce9ef189579bd
I20260812 06:56:46.020754 349 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.52.65:43711 every 8 connection(s)
Aug 12 06:56:46 dist-test-slave-65mx krb5kdc[32059](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517805, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.52.65@KRBTEST.COM for kudu/127.19.52.126@KRBTEST.COM
I20260812 06:56:46.043780 354 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:56:46.020443 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.52.65:50237 (local address 127.19.52.126:37035)
0812 06:56:46.021376 (+ 933us) server_negotiation.cc:206] Beginning negotiation
0812 06:56:46.021383 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 06:56:46.025353 (+ 3970us) server_negotiation.cc:407] Connection header received
0812 06:56:46.025416 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:56:46.025418 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:56:46.025471 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:56:46.025589 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:56:46.027142 (+ 1553us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:46.027700 (+ 558us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:46.028987 (+ 1287us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:46.029226 (+ 239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:46.029342 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:56:46.038513 (+ 9171us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:56:46.038518 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:56:46.038522 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:56:46.038559 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:56:46.040708 (+ 2149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:56:46.041568 (+ 860us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:56:46.041573 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:56:46.041575 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:56:46.041680 (+ 105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:56:46.042237 (+ 557us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:56:46.042241 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:56:46.042243 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:56:46.042501 (+ 258us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:56:46.042596 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0812 06:56:46.043394 (+ 798us) server_negotiation.cc:299] Negotiation successful
0812 06:56:46.043562 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":828,"thread_start_us":128,"threads_started":1}
I20260812 06:56:46.045703 352 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:56:46.019320 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.126:37035 (local address 127.19.52.65:50237)
0812 06:56:46.023632 (+ 4312us) negotiation.cc:107] Waiting for socket to connect
0812 06:56:46.023699 (+ 67us) client_negotiation.cc:174] Beginning negotiation
0812 06:56:46.024981 (+ 1282us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:56:46.025830 (+ 849us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:56:46.025840 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:56:46.026296 (+ 456us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:56:46.026963 (+ 667us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:56:46.026978 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:56:46.027839 (+ 861us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:56:46.027843 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:56:46.028846 (+ 1003us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:56:46.028857 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:56:46.029394 (+ 537us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:56:46.029399 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:56:46.029583 (+ 184us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:56:46.036135 (+ 6552us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 06:56:46.036164 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:56:46.038193 (+ 2029us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 06:56:46.040917 (+ 2724us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:56:46.040925 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:56:46.040947 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:56:46.041410 (+ 463us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:56:46.041878 (+ 468us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:56:46.041881 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:56:46.041884 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:56:46.042113 (+ 229us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:56:46.042679 (+ 566us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 06:56:46.042687 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 06:56:46.043072 (+ 385us) client_negotiation.cc:769] Sending connection context
0812 06:56:46.044084 (+ 1012us) client_negotiation.cc:240] Negotiation successful
0812 06:56:46.044486 (+ 402us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4056,"thread_start_us":162,"threads_started":1}
WARNING: no policy specified for kudu/127.19.52.66@KRBTEST.COM; defaulting to no policy
I20260812 06:56:46.047283 350 heartbeater.cc:347] Connected to a master server at 127.19.52.126:37035
I20260812 06:56:46.047700 350 heartbeater.cc:464] Registering TS with master...
I20260812 06:56:46.048375 350 heartbeater.cc:511] Master 127.19.52.126:37035 requested a full tablet report, sending...
I20260812 06:56:46.050324 32581 ts_manager.cc:194] Registered new tserver with Master: c0dd0a83e72848edb10b2662b59172b3 (127.19.52.65:43711)
I20260812 06:56:46.051970 32581 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.52.65@KRBTEST.COM'} at 127.19.52.65:50237
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.52.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.52.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786517687384214-19665-0/krb5kdc/kudu_127.19.52.66.keytab.
Entry for principal kudu/127.19.52.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786517687384214-19665-0/krb5kdc/kudu_127.19.52.66.keytab.
I20260812 06:56:46.085026 19665 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu
/tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786517687384214-19665-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786517687384214-19665-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786517687384214-19665-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786517687384214-19665-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.19.52.84:60953/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.52.66:0
--local_ip_for_outbound_sockets=127.19.52.66
--webserver_interface=127.19.52.66
--webserver_port=0
--tserver_master_addrs=127.19.52.126:37035
--keytab_file=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786517687384214-19665-0/krb5kdc/kudu_127.19.52.66.keytab
--principal=kudu/127.19.52.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.52.84:40013
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786517687384214-19665-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786517687384214-19665-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786517687384214-19665-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 12 06:56:46 dist-test-slave-65mx krb5kdc[32059](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786517806, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.52.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260812 06:56:46.267751 358 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:56:46.268177 358 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:56:46.268251 358 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 06:56:46.274927 358 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 06:56:46.275034 358 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 06:56:46.275143 358 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.52.66
I20260812 06:56:46.284024 358 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.52.84:40013
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786517687384214-19665-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786517687384214-19665-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.19.52.84:60953/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786517687384214-19665-0/krb5kdc/kudu_127.19.52.66.keytab
--principal=kudu/127.19.52.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.52.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786517687384214-19665-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.19.52.66
--webserver_port=0
--tserver_master_addrs=127.19.52.126:37035
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.358
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.52.66
--log_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786517687384214-19665-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 06:15:54 UTC on ccycloud.quasar-rnfzce.root.comops.site
build id 4581989
I20260812 06:56:46.285655 358 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:56:46.287060 358 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 06:56:46.300184 365 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:56:46.300431 369 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:56:46.300825 366 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:56:46.301049 358 server_base.cc:1034] running on GCE node
I20260812 06:56:46.301637 358 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:56:46.302567 358 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:56:46.303783 358 hybrid_clock.cc:648] HybridClock initialized: now 1786517806303758 us; error 42 us; skew 500 ppm
Aug 12 06:56:46 dist-test-slave-65mx krb5kdc[32059](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517806, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.52.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 06:56:46.308001 358 init.cc:377] Logged in from keytab as kudu/127.19.52.66@KRBTEST.COM (short username kudu)
I20260812 06:56:46.309563 358 webserver.cc:460] Webserver started at http://127.19.52.66:42787/ using document root <none> and password file <none>
I20260812 06:56:46.310489 358 fs_manager.cc:359] Metadata directory not provided
I20260812 06:56:46.310617 358 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:56:46.310904 358 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 12 06:56:46 dist-test-slave-65mx krb5kdc[32059](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517806, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.52.66@KRBTEST.COM for HTTP/127.19.52.84@KRBTEST.COM
I20260812 06:56:46.343660 358 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786517687384214-19665-0/minicluster-data/ts-1/data/instance:
uuid: "2409e68a97f24b9ba2064ec06b670c5e"
format_stamp: "Formatted at 2026-08-12 06:56:46 on dist-test-slave-65mx"
server_key: "ac2a5c974c4987a818b71f85ea1e6a9d"
server_key_iv: "6e2a097c95bfea4f3144ab780db2346c"
server_key_version: "kuduclusterkey@0"
I20260812 06:56:46.348258 358 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786517687384214-19665-0/minicluster-data/ts-1/wal/instance:
uuid: "2409e68a97f24b9ba2064ec06b670c5e"
format_stamp: "Formatted at 2026-08-12 06:56:46 on dist-test-slave-65mx"
server_key: "ac2a5c974c4987a818b71f85ea1e6a9d"
server_key_iv: "6e2a097c95bfea4f3144ab780db2346c"
server_key_version: "kuduclusterkey@0"
I20260812 06:56:46.353893 358 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.003s
I20260812 06:56:46.374275 376 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 06:56:46.376312 358 fs_manager.cc:725] Time spent opening block manager: real 0.006s user 0.003s sys 0.001s
I20260812 06:56:46.376464 358 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786517687384214-19665-0/minicluster-data/ts-1/data,/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786517687384214-19665-0/minicluster-data/ts-1/wal
uuid: "2409e68a97f24b9ba2064ec06b670c5e"
format_stamp: "Formatted at 2026-08-12 06:56:46 on dist-test-slave-65mx"
server_key: "ac2a5c974c4987a818b71f85ea1e6a9d"
server_key_iv: "6e2a097c95bfea4f3144ab780db2346c"
server_key_version: "kuduclusterkey@0"
I20260812 06:56:46.376689 358 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786517687384214-19665-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786517687384214-19665-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786517687384214-19665-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:56:46.398892 358 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 06:56:46.402931 358 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 06:56:46.403147 358 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:56:46.403852 358 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 06:56:46.404965 358 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 06:56:46.405031 358 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 06:56:46.405100 358 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 06:56:46.405130 358 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 06:56:46.424118 358 rpc_server.cc:307] RPC server started. Bound to: 127.19.52.66:46007
I20260812 06:56:46.425417 358 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786517687384214-19665-0/minicluster-data/ts-1/data/info.pb
I20260812 06:56:46.427795 501 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.52.66:46007 every 8 connection(s)
I20260812 06:56:46.431373 19665 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu as pid 358
I20260812 06:56:46.431528 19665 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786517687384214-19665-0/minicluster-data/ts-1/wal/instance
I20260812 06:56:46.431860 19665 external_mini_cluster.cc:1428] Setting key 860076bd6663ad82329d35afc03440b7
WARNING: no policy specified for kudu/127.19.52.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.52.67@KRBTEST.COM" created.
Aug 12 06:56:46 dist-test-slave-65mx krb5kdc[32059](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517806, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.52.66@KRBTEST.COM for kudu/127.19.52.126@KRBTEST.COM
I20260812 06:56:46.459404 354 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:56:46.441878 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.52.66:34595 (local address 127.19.52.126:37035)
0812 06:56:46.442030 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0812 06:56:46.442034 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 06:56:46.444220 (+ 2186us) server_negotiation.cc:407] Connection header received
0812 06:56:46.445791 (+ 1571us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:56:46.445796 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:56:46.445870 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:56:46.445933 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:56:46.447830 (+ 1897us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:46.448485 (+ 655us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:46.449805 (+ 1320us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:46.450224 (+ 419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:46.450350 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:56:46.453956 (+ 3606us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:56:46.453962 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:56:46.453966 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:56:46.454005 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:56:46.456040 (+ 2035us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:56:46.456909 (+ 869us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:56:46.456932 (+ 23us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:56:46.456937 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:56:46.457054 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:56:46.457548 (+ 494us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:56:46.457552 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:56:46.457554 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:56:46.457863 (+ 309us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:56:46.457967 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0812 06:56:46.458999 (+ 1032us) server_negotiation.cc:299] Negotiation successful
0812 06:56:46.459203 (+ 204us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260812 06:56:46.461403 505 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:56:46.442474 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.126:37035 (local address 127.19.52.66:34595)
0812 06:56:46.443991 (+ 1517us) negotiation.cc:107] Waiting for socket to connect
0812 06:56:46.444069 (+ 78us) client_negotiation.cc:174] Beginning negotiation
0812 06:56:46.445414 (+ 1345us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:56:46.446479 (+ 1065us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:56:46.446493 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:56:46.446967 (+ 474us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:56:46.447662 (+ 695us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:56:46.447678 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:56:46.448673 (+ 995us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:56:46.448677 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:56:46.449667 (+ 990us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:56:46.449678 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:56:46.450595 (+ 917us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:56:46.450600 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:56:46.450896 (+ 296us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:56:46.451683 (+ 787us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 06:56:46.451708 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:56:46.453720 (+ 2012us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 06:56:46.456226 (+ 2506us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:56:46.456237 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:56:46.456258 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:56:46.456767 (+ 509us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:56:46.457204 (+ 437us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:56:46.457208 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:56:46.457210 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:56:46.457417 (+ 207us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:56:46.458257 (+ 840us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 06:56:46.458266 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 06:56:46.458667 (+ 401us) client_negotiation.cc:769] Sending connection context
0812 06:56:46.459687 (+ 1020us) client_negotiation.cc:240] Negotiation successful
0812 06:56:46.460105 (+ 418us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1243,"thread_start_us":208,"threads_started":1}
I20260812 06:56:46.462777 502 heartbeater.cc:347] Connected to a master server at 127.19.52.126:37035
I20260812 06:56:46.463037 502 heartbeater.cc:464] Registering TS with master...
I20260812 06:56:46.463518 502 heartbeater.cc:511] Master 127.19.52.126:37035 requested a full tablet report, sending...
I20260812 06:56:46.464614 32581 ts_manager.cc:194] Registered new tserver with Master: 2409e68a97f24b9ba2064ec06b670c5e (127.19.52.66:46007)
I20260812 06:56:46.465245 32581 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.52.66@KRBTEST.COM'} at 127.19.52.66:34595
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.52.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786517687384214-19665-0/krb5kdc/kudu_127.19.52.67.keytab.
Entry for principal kudu/127.19.52.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786517687384214-19665-0/krb5kdc/kudu_127.19.52.67.keytab.
I20260812 06:56:46.496639 19665 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu
/tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786517687384214-19665-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786517687384214-19665-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786517687384214-19665-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786517687384214-19665-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.19.52.84:60953/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.52.67:0
--local_ip_for_outbound_sockets=127.19.52.67
--webserver_interface=127.19.52.67
--webserver_port=0
--tserver_master_addrs=127.19.52.126:37035
--keytab_file=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786517687384214-19665-0/krb5kdc/kudu_127.19.52.67.keytab
--principal=kudu/127.19.52.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.52.84:40013
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786517687384214-19665-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786517687384214-19665-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786517687384214-19665-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 06:56:46.687104 507 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:56:46.687587 507 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:56:46.687726 507 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 06:56:46.694207 507 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 06:56:46.694387 507 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 06:56:46.694543 507 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.52.67
I20260812 06:56:46.703382 507 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.52.84:40013
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786517687384214-19665-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786517687384214-19665-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.19.52.84:60953/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786517687384214-19665-0/krb5kdc/kudu_127.19.52.67.keytab
--principal=kudu/127.19.52.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.52.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786517687384214-19665-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.19.52.67
--webserver_port=0
--tserver_master_addrs=127.19.52.126:37035
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.507
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.52.67
--log_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786517687384214-19665-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 06:15:54 UTC on ccycloud.quasar-rnfzce.root.comops.site
build id 4581989
I20260812 06:56:46.705401 507 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:56:46.707084 507 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 06:56:46.722172 514 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:56:46.723362 516 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:56:46.724412 507 server_base.cc:1034] running on GCE node
W20260812 06:56:46.722235 513 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
Aug 12 06:56:46 dist-test-slave-65mx krb5kdc[32059](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517806, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.52.126@KRBTEST.COM for HTTP/127.19.52.84@KRBTEST.COM
I20260812 06:56:46.726123 507 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:56:46.726920 507 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:56:46.745781 507 hybrid_clock.cc:648] HybridClock initialized: now 1786517806745654 us; error 115 us; skew 500 ppm
Aug 12 06:56:46 dist-test-slave-65mx krb5kdc[32059](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517806, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.52.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 06:56:46.751601 507 init.cc:377] Logged in from keytab as kudu/127.19.52.67@KRBTEST.COM (short username kudu)
I20260812 06:56:46.753031 507 webserver.cc:460] Webserver started at http://127.19.52.67:36317/ using document root <none> and password file <none>
I20260812 06:56:46.753890 507 fs_manager.cc:359] Metadata directory not provided
I20260812 06:56:46.754056 507 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:56:46.754359 507 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 12 06:56:46 dist-test-slave-65mx krb5kdc[32059](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517806, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.52.67@KRBTEST.COM for HTTP/127.19.52.84@KRBTEST.COM
I20260812 06:56:46.784248 507 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786517687384214-19665-0/minicluster-data/ts-2/data/instance:
uuid: "376bc7ecc78040e89a392bd8c03c1e94"
format_stamp: "Formatted at 2026-08-12 06:56:46 on dist-test-slave-65mx"
server_key: "6ab950ffd4403afcbf1343179851ad10"
server_key_iv: "ca6b8293612994f36440bd1ef9e494cf"
server_key_version: "kuduclusterkey@0"
I20260812 06:56:46.785419 507 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786517687384214-19665-0/minicluster-data/ts-2/wal/instance:
uuid: "376bc7ecc78040e89a392bd8c03c1e94"
format_stamp: "Formatted at 2026-08-12 06:56:46 on dist-test-slave-65mx"
server_key: "6ab950ffd4403afcbf1343179851ad10"
server_key_iv: "ca6b8293612994f36440bd1ef9e494cf"
server_key_version: "kuduclusterkey@0"
I20260812 06:56:46.798311 507 fs_manager.cc:691] Time spent creating directory manager: real 0.012s user 0.005s sys 0.000s
I20260812 06:56:46.848637 524 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 06:56:46.854569 507 fs_manager.cc:725] Time spent opening block manager: real 0.014s user 0.004s sys 0.000s
I20260812 06:56:46.854753 507 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786517687384214-19665-0/minicluster-data/ts-2/data,/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786517687384214-19665-0/minicluster-data/ts-2/wal
uuid: "376bc7ecc78040e89a392bd8c03c1e94"
format_stamp: "Formatted at 2026-08-12 06:56:46 on dist-test-slave-65mx"
server_key: "6ab950ffd4403afcbf1343179851ad10"
server_key_iv: "ca6b8293612994f36440bd1ef9e494cf"
server_key_version: "kuduclusterkey@0"
I20260812 06:56:46.854897 507 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786517687384214-19665-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786517687384214-19665-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786517687384214-19665-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:56:46.913396 507 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 06:56:46.924104 507 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 06:56:46.924372 507 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:56:46.925523 507 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 06:56:46.926746 507 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 06:56:46.926855 507 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 06:56:46.926952 507 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 06:56:46.927018 507 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 06:56:46.943074 507 rpc_server.cc:307] RPC server started. Bound to: 127.19.52.67:36281
I20260812 06:56:46.944911 507 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786517687384214-19665-0/minicluster-data/ts-2/data/info.pb
I20260812 06:56:46.945506 647 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.52.67:36281 every 8 connection(s)
I20260812 06:56:46.954910 19665 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu as pid 507
I20260812 06:56:46.955082 19665 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786517687384214-19665-0/minicluster-data/ts-2/wal/instance
I20260812 06:56:46.955891 19665 external_mini_cluster.cc:1428] Setting key 40937ad5fe6a10d69539693db27b873a
Aug 12 06:56:46 dist-test-slave-65mx krb5kdc[32059](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517806, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.52.67@KRBTEST.COM for kudu/127.19.52.126@KRBTEST.COM
I20260812 06:56:46.961959 354 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:56:46.947749 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.52.67:33305 (local address 127.19.52.126:37035)
0812 06:56:46.947923 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0812 06:56:46.947928 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 06:56:46.949615 (+ 1687us) server_negotiation.cc:407] Connection header received
0812 06:56:46.950632 (+ 1017us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:56:46.950635 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:56:46.950697 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:56:46.950806 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:56:46.952031 (+ 1225us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:46.952655 (+ 624us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:46.953880 (+ 1225us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:46.954127 (+ 247us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:46.954242 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:56:46.957693 (+ 3451us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:56:46.957697 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:56:46.957700 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:56:46.957729 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:56:46.959425 (+ 1696us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:56:46.960030 (+ 605us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:56:46.960035 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:56:46.960037 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:56:46.960128 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:56:46.960536 (+ 408us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:56:46.960568 (+ 32us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:56:46.960572 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:56:46.960776 (+ 204us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:56:46.960884 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0812 06:56:46.961650 (+ 766us) server_negotiation.cc:299] Negotiation successful
0812 06:56:46.961785 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260812 06:56:46.962996 650 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:56:46.948224 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.126:37035 (local address 127.19.52.67:33305)
0812 06:56:46.949436 (+ 1212us) negotiation.cc:107] Waiting for socket to connect
0812 06:56:46.949481 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0812 06:56:46.950398 (+ 917us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:56:46.950997 (+ 599us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:56:46.951004 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:56:46.951306 (+ 302us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:56:46.951857 (+ 551us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:56:46.951871 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:56:46.952785 (+ 914us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:56:46.952790 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:56:46.953741 (+ 951us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:56:46.953750 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:56:46.954301 (+ 551us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:56:46.954305 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:56:46.954727 (+ 422us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:56:46.955583 (+ 856us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 06:56:46.955603 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:56:46.957494 (+ 1891us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 06:56:46.959559 (+ 2065us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:56:46.959565 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:56:46.959584 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:56:46.959880 (+ 296us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:56:46.960226 (+ 346us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:56:46.960230 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:56:46.960232 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:56:46.960449 (+ 217us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:56:46.960884 (+ 435us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 06:56:46.960892 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 06:56:46.961237 (+ 345us) client_negotiation.cc:769] Sending connection context
0812 06:56:46.961610 (+ 373us) client_negotiation.cc:240] Negotiation successful
0812 06:56:46.961973 (+ 363us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1030,"thread_start_us":286,"threads_started":1}
I20260812 06:56:46.964395 648 heartbeater.cc:347] Connected to a master server at 127.19.52.126:37035
I20260812 06:56:46.964679 648 heartbeater.cc:464] Registering TS with master...
I20260812 06:56:46.965276 648 heartbeater.cc:511] Master 127.19.52.126:37035 requested a full tablet report, sending...
I20260812 06:56:46.966480 32581 ts_manager.cc:194] Registered new tserver with Master: 376bc7ecc78040e89a392bd8c03c1e94 (127.19.52.67:36281)
I20260812 06:56:46.967124 32581 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.52.67@KRBTEST.COM'} at 127.19.52.67:33305
I20260812 06:56:46.970657 19665 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 06:56:46.990271 354 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:56:46.972999 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46962 (local address 127.19.52.126:37035)
0812 06:56:46.973184 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0812 06:56:46.973189 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 06:56:46.980367 (+ 7178us) server_negotiation.cc:407] Connection header received
0812 06:56:46.980689 (+ 322us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:56:46.980693 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:56:46.980755 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:56:46.980816 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:56:46.981934 (+ 1118us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:46.982767 (+ 833us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:46.983981 (+ 1214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:46.984308 (+ 327us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:46.984394 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:56:46.986392 (+ 1998us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:56:46.986396 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:56:46.986398 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:56:46.986429 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:56:46.988235 (+ 1806us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:56:46.988827 (+ 592us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:56:46.988831 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:56:46.988833 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:56:46.988929 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:56:46.989246 (+ 317us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:56:46.989249 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:56:46.989251 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:56:46.989486 (+ 235us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:56:46.989611 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0812 06:56:46.989972 (+ 361us) server_negotiation.cc:299] Negotiation successful
0812 06:56:46.990106 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260812 06:56:46.994344 32581 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46962:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 06:56:46.999145 32581 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 06:56:47.034795 664 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:56:47.012996 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.67:36281 (local address 127.0.0.1:38272)
0812 06:56:47.019400 (+ 6404us) negotiation.cc:107] Waiting for socket to connect
0812 06:56:47.019462 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0812 06:56:47.019596 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:56:47.022759 (+ 3163us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:56:47.022765 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:56:47.022789 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 06:56:47.022933 (+ 144us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:56:47.022944 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:56:47.026078 (+ 3134us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:56:47.026082 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:56:47.031283 (+ 5201us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:56:47.031294 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:56:47.034198 (+ 2904us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:56:47.034202 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:56:47.034336 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:56:47.034377 (+ 41us) client_negotiation.cc:769] Sending connection context
0812 06:56:47.034495 (+ 118us) client_negotiation.cc:240] Negotiation successful
0812 06:56:47.034631 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6226,"thread_start_us":208,"threads_started":1}
I20260812 06:56:47.036417 662 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:56:47.018239 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.65:43711 (local address 127.0.0.1:43914)
0812 06:56:47.021155 (+ 2916us) negotiation.cc:107] Waiting for socket to connect
0812 06:56:47.021166 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 06:56:47.021234 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:56:47.023971 (+ 2737us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:56:47.023974 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:56:47.023993 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 06:56:47.024088 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:56:47.024105 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:56:47.029244 (+ 5139us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:56:47.029248 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:56:47.030575 (+ 1327us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:56:47.030588 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:56:47.035966 (+ 5378us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:56:47.035969 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:56:47.036064 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:56:47.036087 (+ 23us) client_negotiation.cc:769] Sending connection context
0812 06:56:47.036191 (+ 104us) client_negotiation.cc:240] Negotiation successful
0812 06:56:47.036272 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2860,"thread_start_us":151,"threads_started":1}
I20260812 06:56:47.037544 667 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:56:47.018705 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43914 (local address 127.19.52.65:43711)
0812 06:56:47.020847 (+ 2142us) server_negotiation.cc:206] Beginning negotiation
0812 06:56:47.020854 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 06:56:47.021316 (+ 462us) server_negotiation.cc:407] Connection header received
0812 06:56:47.021387 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:56:47.021392 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:56:47.021606 (+ 214us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:56:47.023147 (+ 1541us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 06:56:47.024335 (+ 1188us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:47.029078 (+ 4743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:47.031448 (+ 2370us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:47.035661 (+ 4213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:47.035772 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:56:47.035907 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0812 06:56:47.037254 (+ 1347us) server_negotiation.cc:299] Negotiation successful
0812 06:56:47.037381 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2043,"thread_start_us":87,"threads_started":1}
I20260812 06:56:47.037850 666 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:56:47.016324 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.66:46007 (local address 127.0.0.1:49316)
0812 06:56:47.019828 (+ 3504us) negotiation.cc:107] Waiting for socket to connect
0812 06:56:47.019841 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 06:56:47.020959 (+ 1118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:56:47.033536 (+ 12577us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:56:47.033540 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:56:47.033560 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 06:56:47.033680 (+ 120us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:56:47.033686 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:56:47.035030 (+ 1344us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:56:47.035033 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:56:47.036109 (+ 1076us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:56:47.036116 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:56:47.037413 (+ 1297us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:56:47.037416 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:56:47.037505 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:56:47.037563 (+ 58us) client_negotiation.cc:769] Sending connection context
0812 06:56:47.037659 (+ 96us) client_negotiation.cc:240] Negotiation successful
0812 06:56:47.037735 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3443,"thread_start_us":256,"threads_started":1}
I20260812 06:56:47.038416 665 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:56:47.016599 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49316 (local address 127.19.52.66:46007)
0812 06:56:47.032807 (+ 16208us) server_negotiation.cc:206] Beginning negotiation
0812 06:56:47.032815 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 06:56:47.032844 (+ 29us) server_negotiation.cc:407] Connection header received
0812 06:56:47.032935 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:56:47.032942 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:56:47.033210 (+ 268us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:56:47.033382 (+ 172us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 06:56:47.033824 (+ 442us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:47.034892 (+ 1068us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:47.036267 (+ 1375us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:47.037120 (+ 853us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:47.037222 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:56:47.037351 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0812 06:56:47.038166 (+ 815us) server_negotiation.cc:299] Negotiation successful
0812 06:56:47.038276 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16061,"thread_start_us":120,"threads_started":1}
I20260812 06:56:47.039175 663 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:56:47.014181 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38272 (local address 127.19.52.67:36281)
0812 06:56:47.022149 (+ 7968us) server_negotiation.cc:206] Beginning negotiation
0812 06:56:47.022161 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0812 06:56:47.022185 (+ 24us) server_negotiation.cc:407] Connection header received
0812 06:56:47.022258 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:56:47.022262 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:56:47.022429 (+ 167us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:56:47.022573 (+ 144us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 06:56:47.024772 (+ 2199us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:47.025917 (+ 1145us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:47.032926 (+ 7009us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:47.033845 (+ 919us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:47.033950 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:56:47.034125 (+ 175us) server_negotiation.cc:1035] Waiting for connection context
0812 06:56:47.038895 (+ 4770us) server_negotiation.cc:299] Negotiation successful
0812 06:56:47.039015 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":7844,"thread_start_us":216,"threads_started":1}
I20260812 06:56:47.041965 576 tablet_service.cc:1467] Processing CreateTablet for tablet 5861e7d85d98426ab36ec3721a01e97e (DEFAULT_TABLE table=test-table [id=059be3acee1c4d48b5c9a22007394cd4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 06:56:47.041965 429 tablet_service.cc:1467] Processing CreateTablet for tablet 5861e7d85d98426ab36ec3721a01e97e (DEFAULT_TABLE table=test-table [id=059be3acee1c4d48b5c9a22007394cd4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 06:56:47.043143 576 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5861e7d85d98426ab36ec3721a01e97e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 06:56:47.043143 429 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5861e7d85d98426ab36ec3721a01e97e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 06:56:47.055217 350 heartbeater.cc:503] Master 127.19.52.126:37035 was elected leader, sending a full tablet report...
I20260812 06:56:47.055574 32741 tablet_service.cc:1467] Processing CreateTablet for tablet 5861e7d85d98426ab36ec3721a01e97e (DEFAULT_TABLE table=test-table [id=059be3acee1c4d48b5c9a22007394cd4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 06:56:47.059087 32741 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5861e7d85d98426ab36ec3721a01e97e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 06:56:47.059634 668 tablet_bootstrap.cc:492] T 5861e7d85d98426ab36ec3721a01e97e P 2409e68a97f24b9ba2064ec06b670c5e: Bootstrap starting.
I20260812 06:56:47.062650 668 tablet_bootstrap.cc:654] T 5861e7d85d98426ab36ec3721a01e97e P 2409e68a97f24b9ba2064ec06b670c5e: Neither blocks nor log segments found. Creating new log.
I20260812 06:56:47.070654 669 tablet_bootstrap.cc:492] T 5861e7d85d98426ab36ec3721a01e97e P 376bc7ecc78040e89a392bd8c03c1e94: Bootstrap starting.
I20260812 06:56:47.072078 668 log.cc:824] T 5861e7d85d98426ab36ec3721a01e97e P 2409e68a97f24b9ba2064ec06b670c5e: Log is configured to *not* fsync() on all Append() calls
I20260812 06:56:47.074435 669 tablet_bootstrap.cc:654] T 5861e7d85d98426ab36ec3721a01e97e P 376bc7ecc78040e89a392bd8c03c1e94: Neither blocks nor log segments found. Creating new log.
I20260812 06:56:47.076104 670 tablet_bootstrap.cc:492] T 5861e7d85d98426ab36ec3721a01e97e P c0dd0a83e72848edb10b2662b59172b3: Bootstrap starting.
I20260812 06:56:47.076397 668 tablet_bootstrap.cc:492] T 5861e7d85d98426ab36ec3721a01e97e P 2409e68a97f24b9ba2064ec06b670c5e: No bootstrap required, opened a new log
I20260812 06:56:47.076699 668 ts_tablet_manager.cc:1397] T 5861e7d85d98426ab36ec3721a01e97e P 2409e68a97f24b9ba2064ec06b670c5e: Time spent bootstrapping tablet: real 0.017s user 0.004s sys 0.003s
I20260812 06:56:47.075461 669 log.cc:824] T 5861e7d85d98426ab36ec3721a01e97e P 376bc7ecc78040e89a392bd8c03c1e94: Log is configured to *not* fsync() on all Append() calls
I20260812 06:56:47.079263 670 tablet_bootstrap.cc:654] T 5861e7d85d98426ab36ec3721a01e97e P c0dd0a83e72848edb10b2662b59172b3: Neither blocks nor log segments found. Creating new log.
I20260812 06:56:47.080183 670 log.cc:824] T 5861e7d85d98426ab36ec3721a01e97e P c0dd0a83e72848edb10b2662b59172b3: Log is configured to *not* fsync() on all Append() calls
I20260812 06:56:47.082201 668 raft_consensus.cc:348] T 5861e7d85d98426ab36ec3721a01e97e P 2409e68a97f24b9ba2064ec06b670c5e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "376bc7ecc78040e89a392bd8c03c1e94" member_type: VOTER last_known_addr { host: "127.19.52.67" port: 36281 } } peers { permanent_uuid: "2409e68a97f24b9ba2064ec06b670c5e" member_type: VOTER last_known_addr { host: "127.19.52.66" port: 46007 } } peers { permanent_uuid: "c0dd0a83e72848edb10b2662b59172b3" member_type: VOTER last_known_addr { host: "127.19.52.65" port: 43711 } }
I20260812 06:56:47.082664 668 raft_consensus.cc:374] T 5861e7d85d98426ab36ec3721a01e97e P 2409e68a97f24b9ba2064ec06b670c5e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 06:56:47.082773 668 raft_consensus.cc:729] T 5861e7d85d98426ab36ec3721a01e97e P 2409e68a97f24b9ba2064ec06b670c5e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2409e68a97f24b9ba2064ec06b670c5e, State: Initialized, Role: FOLLOWER
I20260812 06:56:47.083463 668 consensus_queue.cc:260] T 5861e7d85d98426ab36ec3721a01e97e P 2409e68a97f24b9ba2064ec06b670c5e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "376bc7ecc78040e89a392bd8c03c1e94" member_type: VOTER last_known_addr { host: "127.19.52.67" port: 36281 } } peers { permanent_uuid: "2409e68a97f24b9ba2064ec06b670c5e" member_type: VOTER last_known_addr { host: "127.19.52.66" port: 46007 } } peers { permanent_uuid: "c0dd0a83e72848edb10b2662b59172b3" member_type: VOTER last_known_addr { host: "127.19.52.65" port: 43711 } }
I20260812 06:56:47.084339 669 tablet_bootstrap.cc:492] T 5861e7d85d98426ab36ec3721a01e97e P 376bc7ecc78040e89a392bd8c03c1e94: No bootstrap required, opened a new log
I20260812 06:56:47.084606 669 ts_tablet_manager.cc:1397] T 5861e7d85d98426ab36ec3721a01e97e P 376bc7ecc78040e89a392bd8c03c1e94: Time spent bootstrapping tablet: real 0.014s user 0.002s sys 0.005s
I20260812 06:56:47.084975 668 ts_tablet_manager.cc:1428] T 5861e7d85d98426ab36ec3721a01e97e P 2409e68a97f24b9ba2064ec06b670c5e: Time spent starting tablet: real 0.008s user 0.006s sys 0.001s
I20260812 06:56:47.085448 502 heartbeater.cc:503] Master 127.19.52.126:37035 was elected leader, sending a full tablet report...
I20260812 06:56:47.088270 670 tablet_bootstrap.cc:492] T 5861e7d85d98426ab36ec3721a01e97e P c0dd0a83e72848edb10b2662b59172b3: No bootstrap required, opened a new log
I20260812 06:56:47.088598 670 ts_tablet_manager.cc:1397] T 5861e7d85d98426ab36ec3721a01e97e P c0dd0a83e72848edb10b2662b59172b3: Time spent bootstrapping tablet: real 0.013s user 0.007s sys 0.000s
I20260812 06:56:47.089013 669 raft_consensus.cc:348] T 5861e7d85d98426ab36ec3721a01e97e P 376bc7ecc78040e89a392bd8c03c1e94 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "376bc7ecc78040e89a392bd8c03c1e94" member_type: VOTER last_known_addr { host: "127.19.52.67" port: 36281 } } peers { permanent_uuid: "2409e68a97f24b9ba2064ec06b670c5e" member_type: VOTER last_known_addr { host: "127.19.52.66" port: 46007 } } peers { permanent_uuid: "c0dd0a83e72848edb10b2662b59172b3" member_type: VOTER last_known_addr { host: "127.19.52.65" port: 43711 } }
I20260812 06:56:47.089310 669 raft_consensus.cc:374] T 5861e7d85d98426ab36ec3721a01e97e P 376bc7ecc78040e89a392bd8c03c1e94 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 06:56:47.089375 669 raft_consensus.cc:729] T 5861e7d85d98426ab36ec3721a01e97e P 376bc7ecc78040e89a392bd8c03c1e94 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 376bc7ecc78040e89a392bd8c03c1e94, State: Initialized, Role: FOLLOWER
I20260812 06:56:47.089896 669 consensus_queue.cc:260] T 5861e7d85d98426ab36ec3721a01e97e P 376bc7ecc78040e89a392bd8c03c1e94 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "376bc7ecc78040e89a392bd8c03c1e94" member_type: VOTER last_known_addr { host: "127.19.52.67" port: 36281 } } peers { permanent_uuid: "2409e68a97f24b9ba2064ec06b670c5e" member_type: VOTER last_known_addr { host: "127.19.52.66" port: 46007 } } peers { permanent_uuid: "c0dd0a83e72848edb10b2662b59172b3" member_type: VOTER last_known_addr { host: "127.19.52.65" port: 43711 } }
I20260812 06:56:47.091096 669 ts_tablet_manager.cc:1428] T 5861e7d85d98426ab36ec3721a01e97e P 376bc7ecc78040e89a392bd8c03c1e94: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260812 06:56:47.091460 648 heartbeater.cc:503] Master 127.19.52.126:37035 was elected leader, sending a full tablet report...
I20260812 06:56:47.093200 670 raft_consensus.cc:348] T 5861e7d85d98426ab36ec3721a01e97e P c0dd0a83e72848edb10b2662b59172b3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "376bc7ecc78040e89a392bd8c03c1e94" member_type: VOTER last_known_addr { host: "127.19.52.67" port: 36281 } } peers { permanent_uuid: "2409e68a97f24b9ba2064ec06b670c5e" member_type: VOTER last_known_addr { host: "127.19.52.66" port: 46007 } } peers { permanent_uuid: "c0dd0a83e72848edb10b2662b59172b3" member_type: VOTER last_known_addr { host: "127.19.52.65" port: 43711 } }
I20260812 06:56:47.093598 670 raft_consensus.cc:374] T 5861e7d85d98426ab36ec3721a01e97e P c0dd0a83e72848edb10b2662b59172b3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 06:56:47.093708 670 raft_consensus.cc:729] T 5861e7d85d98426ab36ec3721a01e97e P c0dd0a83e72848edb10b2662b59172b3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c0dd0a83e72848edb10b2662b59172b3, State: Initialized, Role: FOLLOWER
I20260812 06:56:47.094278 670 consensus_queue.cc:260] T 5861e7d85d98426ab36ec3721a01e97e P c0dd0a83e72848edb10b2662b59172b3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "376bc7ecc78040e89a392bd8c03c1e94" member_type: VOTER last_known_addr { host: "127.19.52.67" port: 36281 } } peers { permanent_uuid: "2409e68a97f24b9ba2064ec06b670c5e" member_type: VOTER last_known_addr { host: "127.19.52.66" port: 46007 } } peers { permanent_uuid: "c0dd0a83e72848edb10b2662b59172b3" member_type: VOTER last_known_addr { host: "127.19.52.65" port: 43711 } }
I20260812 06:56:47.095549 670 ts_tablet_manager.cc:1428] T 5861e7d85d98426ab36ec3721a01e97e P c0dd0a83e72848edb10b2662b59172b3: Time spent starting tablet: real 0.007s user 0.005s sys 0.000s
I20260812 06:56:47.111155 674 raft_consensus.cc:482] T 5861e7d85d98426ab36ec3721a01e97e P 2409e68a97f24b9ba2064ec06b670c5e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 06:56:47.111438 674 raft_consensus.cc:504] T 5861e7d85d98426ab36ec3721a01e97e P 2409e68a97f24b9ba2064ec06b670c5e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "376bc7ecc78040e89a392bd8c03c1e94" member_type: VOTER last_known_addr { host: "127.19.52.67" port: 36281 } } peers { permanent_uuid: "2409e68a97f24b9ba2064ec06b670c5e" member_type: VOTER last_known_addr { host: "127.19.52.66" port: 46007 } } peers { permanent_uuid: "c0dd0a83e72848edb10b2662b59172b3" member_type: VOTER last_known_addr { host: "127.19.52.65" port: 43711 } }
I20260812 06:56:47.112777 674 leader_election.cc:288] T 5861e7d85d98426ab36ec3721a01e97e P 2409e68a97f24b9ba2064ec06b670c5e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 376bc7ecc78040e89a392bd8c03c1e94 (127.19.52.67:36281), c0dd0a83e72848edb10b2662b59172b3 (127.19.52.65:43711)
I20260812 06:56:47.118749 667 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:56:47.113505 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.52.66:44911 (local address 127.19.52.65:43711)
0812 06:56:47.113668 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0812 06:56:47.113673 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 06:56:47.114234 (+ 561us) server_negotiation.cc:407] Connection header received
0812 06:56:47.114418 (+ 184us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:56:47.114421 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:56:47.114471 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:56:47.114522 (+ 51us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 06:56:47.114974 (+ 452us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:47.115692 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:47.117265 (+ 1573us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:47.117826 (+ 561us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:47.117916 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:56:47.117969 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0812 06:56:47.118527 (+ 558us) server_negotiation.cc:299] Negotiation successful
0812 06:56:47.118613 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260812 06:56:47.119504 678 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:56:47.113210 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.65:43711 (local address 127.19.52.66:44911)
0812 06:56:47.114146 (+ 936us) negotiation.cc:107] Waiting for socket to connect
0812 06:56:47.114160 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 06:56:47.114299 (+ 139us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:56:47.114674 (+ 375us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:56:47.114676 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:56:47.114692 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 06:56:47.114840 (+ 148us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:56:47.114847 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:56:47.115839 (+ 992us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:56:47.115842 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:56:47.117122 (+ 1280us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:56:47.117134 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:56:47.118076 (+ 942us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:56:47.118079 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:56:47.118328 (+ 249us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:56:47.118361 (+ 33us) client_negotiation.cc:769] Sending connection context
0812 06:56:47.119199 (+ 838us) client_negotiation.cc:240] Negotiation successful
0812 06:56:47.119285 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":788,"thread_start_us":97,"threads_started":1}
I20260812 06:56:47.119748 663 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:56:47.113510 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.52.66:36763 (local address 127.19.52.67:36281)
0812 06:56:47.113787 (+ 277us) server_negotiation.cc:206] Beginning negotiation
0812 06:56:47.113792 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 06:56:47.113807 (+ 15us) server_negotiation.cc:407] Connection header received
0812 06:56:47.113979 (+ 172us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:56:47.113982 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:56:47.114035 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:56:47.114089 (+ 54us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 06:56:47.114594 (+ 505us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:47.115420 (+ 826us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:47.118341 (+ 2921us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:47.118958 (+ 617us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:47.119125 (+ 167us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:56:47.119172 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0812 06:56:47.119533 (+ 361us) server_negotiation.cc:299] Negotiation successful
0812 06:56:47.119634 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":184}
I20260812 06:56:47.120168 679 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:56:47.112978 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.67:36281 (local address 127.19.52.66:36763)
0812 06:56:47.113697 (+ 719us) negotiation.cc:107] Waiting for socket to connect
0812 06:56:47.113714 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0812 06:56:47.113836 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:56:47.114288 (+ 452us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:56:47.114291 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:56:47.114303 (+ 12us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 06:56:47.114467 (+ 164us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:56:47.114474 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:56:47.115614 (+ 1140us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:56:47.115617 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:56:47.118063 (+ 2446us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:56:47.118070 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:56:47.119268 (+ 1198us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:56:47.119271 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:56:47.119380 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:56:47.119401 (+ 21us) client_negotiation.cc:769] Sending connection context
0812 06:56:47.119938 (+ 537us) client_negotiation.cc:240] Negotiation successful
0812 06:56:47.120032 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":648,"thread_start_us":163,"threads_started":1}
I20260812 06:56:47.120376 32766 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5861e7d85d98426ab36ec3721a01e97e" candidate_uuid: "2409e68a97f24b9ba2064ec06b670c5e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c0dd0a83e72848edb10b2662b59172b3" is_pre_election: true
I20260812 06:56:47.120786 32766 raft_consensus.cc:2454] T 5861e7d85d98426ab36ec3721a01e97e P c0dd0a83e72848edb10b2662b59172b3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2409e68a97f24b9ba2064ec06b670c5e in term 0.
I20260812 06:56:47.120873 596 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5861e7d85d98426ab36ec3721a01e97e" candidate_uuid: "2409e68a97f24b9ba2064ec06b670c5e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "376bc7ecc78040e89a392bd8c03c1e94" is_pre_election: true
I20260812 06:56:47.121186 596 raft_consensus.cc:2454] T 5861e7d85d98426ab36ec3721a01e97e P 376bc7ecc78040e89a392bd8c03c1e94 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2409e68a97f24b9ba2064ec06b670c5e in term 0.
I20260812 06:56:47.121384 379 leader_election.cc:302] T 5861e7d85d98426ab36ec3721a01e97e P 2409e68a97f24b9ba2064ec06b670c5e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2409e68a97f24b9ba2064ec06b670c5e, c0dd0a83e72848edb10b2662b59172b3; no voters:
I20260812 06:56:47.121732 674 raft_consensus.cc:2781] T 5861e7d85d98426ab36ec3721a01e97e P 2409e68a97f24b9ba2064ec06b670c5e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 06:56:47.121836 674 raft_consensus.cc:482] T 5861e7d85d98426ab36ec3721a01e97e P 2409e68a97f24b9ba2064ec06b670c5e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 06:56:47.121919 674 raft_consensus.cc:3037] T 5861e7d85d98426ab36ec3721a01e97e P 2409e68a97f24b9ba2064ec06b670c5e [term 0 FOLLOWER]: Advancing to term 1
I20260812 06:56:47.123359 674 raft_consensus.cc:504] T 5861e7d85d98426ab36ec3721a01e97e P 2409e68a97f24b9ba2064ec06b670c5e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "376bc7ecc78040e89a392bd8c03c1e94" member_type: VOTER last_known_addr { host: "127.19.52.67" port: 36281 } } peers { permanent_uuid: "2409e68a97f24b9ba2064ec06b670c5e" member_type: VOTER last_known_addr { host: "127.19.52.66" port: 46007 } } peers { permanent_uuid: "c0dd0a83e72848edb10b2662b59172b3" member_type: VOTER last_known_addr { host: "127.19.52.65" port: 43711 } }
I20260812 06:56:47.123765 674 leader_election.cc:288] T 5861e7d85d98426ab36ec3721a01e97e P 2409e68a97f24b9ba2064ec06b670c5e [CANDIDATE]: Term 1 election: Requested vote from peers 376bc7ecc78040e89a392bd8c03c1e94 (127.19.52.67:36281), c0dd0a83e72848edb10b2662b59172b3 (127.19.52.65:43711)
I20260812 06:56:47.124161 596 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5861e7d85d98426ab36ec3721a01e97e" candidate_uuid: "2409e68a97f24b9ba2064ec06b670c5e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "376bc7ecc78040e89a392bd8c03c1e94"
I20260812 06:56:47.124287 596 raft_consensus.cc:3037] T 5861e7d85d98426ab36ec3721a01e97e P 376bc7ecc78040e89a392bd8c03c1e94 [term 0 FOLLOWER]: Advancing to term 1
I20260812 06:56:47.124488 32766 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5861e7d85d98426ab36ec3721a01e97e" candidate_uuid: "2409e68a97f24b9ba2064ec06b670c5e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c0dd0a83e72848edb10b2662b59172b3"
I20260812 06:56:47.124639 32766 raft_consensus.cc:3037] T 5861e7d85d98426ab36ec3721a01e97e P c0dd0a83e72848edb10b2662b59172b3 [term 0 FOLLOWER]: Advancing to term 1
I20260812 06:56:47.125845 596 raft_consensus.cc:2454] T 5861e7d85d98426ab36ec3721a01e97e P 376bc7ecc78040e89a392bd8c03c1e94 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2409e68a97f24b9ba2064ec06b670c5e in term 1.
I20260812 06:56:47.126318 377 leader_election.cc:302] T 5861e7d85d98426ab36ec3721a01e97e P 2409e68a97f24b9ba2064ec06b670c5e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2409e68a97f24b9ba2064ec06b670c5e, 376bc7ecc78040e89a392bd8c03c1e94; no voters:
I20260812 06:56:47.126405 32766 raft_consensus.cc:2454] T 5861e7d85d98426ab36ec3721a01e97e P c0dd0a83e72848edb10b2662b59172b3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2409e68a97f24b9ba2064ec06b670c5e in term 1.
I20260812 06:56:47.126690 674 raft_consensus.cc:2781] T 5861e7d85d98426ab36ec3721a01e97e P 2409e68a97f24b9ba2064ec06b670c5e [term 1 FOLLOWER]: Leader election won for term 1
I20260812 06:56:47.126950 674 raft_consensus.cc:686] T 5861e7d85d98426ab36ec3721a01e97e P 2409e68a97f24b9ba2064ec06b670c5e [term 1 LEADER]: Becoming Leader. State: Replica: 2409e68a97f24b9ba2064ec06b670c5e, State: Running, Role: LEADER
I20260812 06:56:47.127370 674 consensus_queue.cc:237] T 5861e7d85d98426ab36ec3721a01e97e P 2409e68a97f24b9ba2064ec06b670c5e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "376bc7ecc78040e89a392bd8c03c1e94" member_type: VOTER last_known_addr { host: "127.19.52.67" port: 36281 } } peers { permanent_uuid: "2409e68a97f24b9ba2064ec06b670c5e" member_type: VOTER last_known_addr { host: "127.19.52.66" port: 46007 } } peers { permanent_uuid: "c0dd0a83e72848edb10b2662b59172b3" member_type: VOTER last_known_addr { host: "127.19.52.65" port: 43711 } }
I20260812 06:56:47.131273 32581 catalog_manager.cc:5495] T 5861e7d85d98426ab36ec3721a01e97e P 2409e68a97f24b9ba2064ec06b670c5e reported cstate change: term changed from 0 to 1, leader changed from <none> to 2409e68a97f24b9ba2064ec06b670c5e (127.19.52.66). New cstate: current_term: 1 leader_uuid: "2409e68a97f24b9ba2064ec06b670c5e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "376bc7ecc78040e89a392bd8c03c1e94" member_type: VOTER last_known_addr { host: "127.19.52.67" port: 36281 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2409e68a97f24b9ba2064ec06b670c5e" member_type: VOTER last_known_addr { host: "127.19.52.66" port: 46007 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c0dd0a83e72848edb10b2662b59172b3" member_type: VOTER last_known_addr { host: "127.19.52.65" port: 43711 } health_report { overall_health: UNKNOWN } } }
I20260812 06:56:47.154390 665 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:56:47.148655 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49326 (local address 127.19.52.66:46007)
0812 06:56:47.148785 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0812 06:56:47.148790 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 06:56:47.148809 (+ 19us) server_negotiation.cc:407] Connection header received
0812 06:56:47.149039 (+ 230us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:56:47.149043 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:56:47.149127 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:56:47.149252 (+ 125us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 06:56:47.149675 (+ 423us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:47.150485 (+ 810us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:47.151833 (+ 1348us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:47.152195 (+ 362us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:47.152311 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:56:47.152828 (+ 517us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 06:56:47.153463 (+ 635us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 06:56:47.153600 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0812 06:56:47.154156 (+ 556us) server_negotiation.cc:299] Negotiation successful
0812 06:56:47.154234 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260812 06:56:47.162472 32766 raft_consensus.cc:1261] T 5861e7d85d98426ab36ec3721a01e97e P c0dd0a83e72848edb10b2662b59172b3 [term 1 FOLLOWER]: Refusing update from remote peer 2409e68a97f24b9ba2064ec06b670c5e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 06:56:47.163720 596 raft_consensus.cc:1261] T 5861e7d85d98426ab36ec3721a01e97e P 376bc7ecc78040e89a392bd8c03c1e94 [term 1 FOLLOWER]: Refusing update from remote peer 2409e68a97f24b9ba2064ec06b670c5e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 06:56:47.164705 680 consensus_queue.cc:1035] T 5861e7d85d98426ab36ec3721a01e97e P 2409e68a97f24b9ba2064ec06b670c5e [LEADER]: Connected to new peer: Peer: permanent_uuid: "c0dd0a83e72848edb10b2662b59172b3" member_type: VOTER last_known_addr { host: "127.19.52.65" port: 43711 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 06:56:47.165027 674 consensus_queue.cc:1035] T 5861e7d85d98426ab36ec3721a01e97e P 2409e68a97f24b9ba2064ec06b670c5e [LEADER]: Connected to new peer: Peer: permanent_uuid: "376bc7ecc78040e89a392bd8c03c1e94" member_type: VOTER last_known_addr { host: "127.19.52.67" port: 36281 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260812 06:56:47.178930 503 tablet.cc:2369] T 5861e7d85d98426ab36ec3721a01e97e P 2409e68a97f24b9ba2064ec06b670c5e: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 06:56:47.179915 684 mvcc.cc:205] Tried to move back new op lower bound from 7317576938126127104 to 7317576937997352960. Current Snapshot: MvccSnapshot[applied={T|T < 7317576938126127104}]
Aug 12 06:56:47 dist-test-slave-65mx krb5kdc[32059](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517806, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.52.126@KRBTEST.COM for HTTP/127.19.52.84@KRBTEST.COM
I20260812 06:56:47.187863 685 mvcc.cc:205] Tried to move back new op lower bound from 7317576938126127104 to 7317576937997352960. Current Snapshot: MvccSnapshot[applied={T|T < 7317576938126127104}]
I20260812 06:56:47.188798 686 mvcc.cc:205] Tried to move back new op lower bound from 7317576938126127104 to 7317576937997352960. Current Snapshot: MvccSnapshot[applied={T|T < 7317576938126127104}]
I20260812 06:56:47.198830 32581 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46962:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 06:56:47.199024 32581 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46962:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 06:56:47.202539 32581 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 9c57f5da932a430c874acc23a69c8fbe: Sending DeleteTablet for 3 replicas of tablet 5861e7d85d98426ab36ec3721a01e97e
I20260812 06:56:47.203933 32741 tablet_service.cc:1514] Processing DeleteTablet for tablet 5861e7d85d98426ab36ec3721a01e97e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 06:56:47 UTC) from {username='kudu', principal='kudu/127.19.52.126@KRBTEST.COM'} at 127.0.0.1:43914
I20260812 06:56:47.205122 429 tablet_service.cc:1514] Processing DeleteTablet for tablet 5861e7d85d98426ab36ec3721a01e97e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 06:56:47 UTC) from {username='kudu', principal='kudu/127.19.52.126@KRBTEST.COM'} at 127.0.0.1:49316
I20260812 06:56:47.206105 576 tablet_service.cc:1514] Processing DeleteTablet for tablet 5861e7d85d98426ab36ec3721a01e97e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 06:56:47 UTC) from {username='kudu', principal='kudu/127.19.52.126@KRBTEST.COM'} at 127.0.0.1:38272
I20260812 06:56:47.207060 693 tablet_replica.cc:330] T 5861e7d85d98426ab36ec3721a01e97e P c0dd0a83e72848edb10b2662b59172b3: stopping tablet replica
I20260812 06:56:47.207381 19665 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu with pid 32670
I20260812 06:56:47.208915 694 tablet_replica.cc:330] T 5861e7d85d98426ab36ec3721a01e97e P 2409e68a97f24b9ba2064ec06b670c5e: stopping tablet replica
I20260812 06:56:47.209059 695 tablet_replica.cc:330] T 5861e7d85d98426ab36ec3721a01e97e P 376bc7ecc78040e89a392bd8c03c1e94: stopping tablet replica
I20260812 06:56:47.209569 694 raft_consensus.cc:2229] T 5861e7d85d98426ab36ec3721a01e97e P 2409e68a97f24b9ba2064ec06b670c5e [term 1 LEADER]: Raft consensus shutting down.
I20260812 06:56:47.209998 695 raft_consensus.cc:2229] T 5861e7d85d98426ab36ec3721a01e97e P 376bc7ecc78040e89a392bd8c03c1e94 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 06:56:47.210592 695 raft_consensus.cc:2258] T 5861e7d85d98426ab36ec3721a01e97e P 376bc7ecc78040e89a392bd8c03c1e94 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 06:56:47.211076 694 raft_consensus.cc:2258] T 5861e7d85d98426ab36ec3721a01e97e P 2409e68a97f24b9ba2064ec06b670c5e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 06:56:47.213207 694 ts_tablet_manager.cc:1905] T 5861e7d85d98426ab36ec3721a01e97e P 2409e68a97f24b9ba2064ec06b670c5e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 06:56:47.214931 695 ts_tablet_manager.cc:1905] T 5861e7d85d98426ab36ec3721a01e97e P 376bc7ecc78040e89a392bd8c03c1e94: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 06:56:47.219426 694 ts_tablet_manager.cc:1918] T 5861e7d85d98426ab36ec3721a01e97e P 2409e68a97f24b9ba2064ec06b670c5e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
W20260812 06:56:47.220348 32567 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.19.52.65:43711 (error 108)
I20260812 06:56:47.220623 19665 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu with pid 358
I20260812 06:56:47.225739 666 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:56:47.225334 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.65:43711 (local address 127.0.0.1:43918)
0812 06:56:47.225547 (+ 213us) negotiation.cc:107] Waiting for socket to connect
0812 06:56:47.225628 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.52.65:43711: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":129}
W20260812 06:56:47.226092 32567 catalog_manager.cc:4584] TS c0dd0a83e72848edb10b2662b59172b3 (127.19.52.65:43711): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 5861e7d85d98426ab36ec3721a01e97e: Network error: Client connection negotiation failed: client connection to 127.19.52.65:43711: connect: Connection refused (error 111)
I20260812 06:56:47.226768 695 ts_tablet_manager.cc:1918] T 5861e7d85d98426ab36ec3721a01e97e P 376bc7ecc78040e89a392bd8c03c1e94: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 06:56:47.226955 695 log.cc:1196] T 5861e7d85d98426ab36ec3721a01e97e P 376bc7ecc78040e89a392bd8c03c1e94: Deleting WAL directory at /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786517687384214-19665-0/minicluster-data/ts-2/wal/wals/5861e7d85d98426ab36ec3721a01e97e
I20260812 06:56:47.227475 695 ts_tablet_manager.cc:1939] T 5861e7d85d98426ab36ec3721a01e97e P 376bc7ecc78040e89a392bd8c03c1e94: Deleting consensus metadata
I20260812 06:56:47.229882 32565 catalog_manager.cc:4841] TS 376bc7ecc78040e89a392bd8c03c1e94 (127.19.52.67:36281): tablet 5861e7d85d98426ab36ec3721a01e97e (table test-table [id=059be3acee1c4d48b5c9a22007394cd4]) successfully deleted
I20260812 06:56:47.231895 19665 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu with pid 507
I20260812 06:56:47.234239 666 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:56:47.233879 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.66:46007 (local address 127.0.0.1:49340)
0812 06:56:47.234082 (+ 203us) negotiation.cc:107] Waiting for socket to connect
0812 06:56:47.234142 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.52.66:46007: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":113}
I20260812 06:56:47.242256 19665 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu with pid 32549
I20260812 06:56:47.263212 19665 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260812 06:56:47.609648 19665 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260812 06:56:47.609784 19665 mini_ranger.cc:66] Stopping Ranger...
I20260812 06:56:47.966179 19665 mini_ranger.cc:68] Stopped Ranger
2026-08-12 06:56:47.966 UTC [32091] LOG: received smart shutdown request
2026-08-12 06:56:47.971 UTC [32091] LOG: background worker "logical replication launcher" (PID 32100) exited with exit code 1
2026-08-12 06:56:47.971 UTC [32095] LOG: shutting down
2026-08-12 06:56:48.194 UTC [32091] LOG: database system is shut down
2026-08-12T06:56:48Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegration (51491 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers
Loading random data
Initializing database '/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517687384214-19665-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 06:56:48 dist-test-slave-65mx krb5kdc[707](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 06:56:48 dist-test-slave-65mx krb5kdc[707](info): set up 2 sockets
Aug 12 06:56:48 dist-test-slave-65mx krb5kdc[707](info): commencing operation
krb5kdc: starting...
W20260812 06:56:49.479480 19665 mini_kdc.cc:121] Time spent starting KDC: real 1.122s user 0.004s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517687384214-19665-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517687384214-19665-0/krb5kdc/test-user.keytab.
Aug 12 06:56:49 dist-test-slave-65mx krb5kdc[707](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517809, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T06:56:49Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T06:56:49Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.19.52.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.19.52.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.19.52.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517687384214-19665-0/krb5kdc/rangeradmin_127.19.52.84@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.19.52.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517687384214-19665-0/krb5kdc/rangeradmin_127.19.52.84@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.19.52.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.19.52.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.19.52.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517687384214-19665-0/krb5kdc/rangerlookup_127.19.52.84@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.19.52.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517687384214-19665-0/krb5kdc/rangerlookup_127.19.52.84@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.19.52.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.19.52.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.19.52.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517687384214-19665-0/krb5kdc/HTTP_127.19.52.84@KRBTEST.COM.keytab.
Entry for principal HTTP/127.19.52.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517687384214-19665-0/krb5kdc/HTTP_127.19.52.84@KRBTEST.COM.keytab.
I20260812 06:56:49.687573 19665 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517687384214-19665-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskG7Vh1a/build/debug/bin/postgres/pg_ctl -D /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517687384214-19665-0/minicluster-data/postgres -l logfile start
2026-08-12 06:56:52.520 UTC [739] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-12 06:56:52.520 UTC [739] LOG: listening on IPv4 address "127.19.52.84", port 57645
2026-08-12 06:56:52.524 UTC [739] LOG: listening on Unix socket "/tmp/.s.PGSQL.57645"
2026-08-12 06:56:52.548 UTC [742] LOG: database system was shut down at 2026-08-12 06:56:50 UTC
2026-08-12 06:56:52.552 UTC [739] LOG: database system is ready to accept connections
I20260812 06:56:53.566455 19665 mini_postgres.cc:96] Postgres bound to 57645
2026-08-12 06:56:53.574 UTC [751] FATAL: database "slave" does not exist
127.19.52.84:57645 - accepting connections
I20260812 06:56:53.575018 19665 mini_ranger.cc:161] Starting Ranger...
I20260812 06:56:53.592789 19665 mini_ranger.cc:84] Created miniranger Postgres user
I20260812 06:56:54.161602 19665 mini_ranger.cc:87] Created ranger Postgres database
I20260812 06:56:54.161701 19665 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517687384214-19665-0/minicluster-data/ranger-admin
2026-08-12 06:56:54,453 [I] DB FLAVOR :POSTGRES
2026-08-12 06:56:54,454 [I] --------- Verifying Ranger DB connection ---------
2026-08-12 06:56:54,454 [I] Checking connection..
2026-08-12 06:56:54,454 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskG7Vh1a/build/debug/bin/postgresql.jar:/tmp/dist-test-taskG7Vh1a/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.19.52.84:57645/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-12 06:56:54,732 [I] Checking connection passed.
2026-08-12 06:56:54,732 [I] --------- Verifying version history table ---------
2026-08-12 06:56:54,732 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskG7Vh1a/build/debug/bin/postgresql.jar:/tmp/dist-test-taskG7Vh1a/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.19.52.84:57645/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-12 06:56:55,025 [I] Table x_db_version_h does not exist in database ranger
2026-08-12 06:56:55,025 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskG7Vh1a/build/debug/bin/postgresql.jar:/tmp/dist-test-taskG7Vh1a/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.19.52.84:57645/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-12 06:56:55,309 [I] Table x_db_version_h does not exist in database ranger
2026-08-12 06:56:55,309 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-12 06:56:55,310 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskG7Vh1a/build/debug/bin/postgresql.jar:/tmp/dist-test-taskG7Vh1a/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.19.52.84:57645/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskG7Vh1a/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-12 06:56:55.592 UTC [822] WARNING: there is no transaction in progress
2026-08-12 06:56:55,599 [I] create_dbversion_catalog.sql file imported successfully
2026-08-12 06:56:55,600 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskG7Vh1a/build/debug/bin/postgresql.jar:/tmp/dist-test-taskG7Vh1a/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.19.52.84:57645/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-12 06:56:55,877 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-12 06:56:55,877 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-12 06:56:55,877 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskG7Vh1a/build/debug/bin/postgresql.jar:/tmp/dist-test-taskG7Vh1a/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.19.52.84:57645/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-12 06:56:56,139 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskG7Vh1a/build/debug/bin/postgresql.jar:/tmp/dist-test-taskG7Vh1a/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.19.52.84:57645/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-12 06:56:56,399 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskG7Vh1a/build/debug/bin/postgresql.jar:/tmp/dist-test-taskG7Vh1a/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.19.52.84:57645/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-65mx.c.gcp-upstream.internal','N') ;"
2026-08-12 06:56:56,651 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskG7Vh1a/build/debug/bin/postgresql.jar:/tmp/dist-test-taskG7Vh1a/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.19.52.84:57645/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-12 06:56:56,897 [I] Table x_portal_user does not exist in database ranger
2026-08-12 06:56:56,898 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskG7Vh1a/build/debug/bin/postgresql.jar:/tmp/dist-test-taskG7Vh1a/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.19.52.84:57645/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-12 06:56:57,156 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-12 06:56:57,157 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskG7Vh1a/build/debug/bin/postgresql.jar:/tmp/dist-test-taskG7Vh1a/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.19.52.84:57645/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-12 06:56:57,406 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskG7Vh1a/build/debug/bin/postgresql.jar:/tmp/dist-test-taskG7Vh1a/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.19.52.84:57645/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-12 06:56:57,686 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-12 06:56:57,687 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskG7Vh1a/build/debug/bin/postgresql.jar:/tmp/dist-test-taskG7Vh1a/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.19.52.84:57645/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskG7Vh1a/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-12 06:56:58.864 UTC [941] WARNING: there is no transaction in progress
2026-08-12 06:56:58.884 UTC [941] WARNING: there is no transaction in progress
2026-08-12 06:56:58.906 UTC [941] WARNING: there is no transaction in progress
2026-08-12 06:56:58.926 UTC [941] WARNING: there is no transaction in progress
2026-08-12 06:56:58.948 UTC [941] WARNING: there is no transaction in progress
2026-08-12 06:56:58.973 UTC [941] WARNING: there is no transaction in progress
2026-08-12 06:56:59.040 UTC [941] WARNING: there is no transaction in progress
2026-08-12 06:56:59.049 UTC [941] WARNING: there is no transaction in progress
2026-08-12 06:56:59.067 UTC [941] WARNING: there is no transaction in progress
2026-08-12 06:56:59.080 UTC [941] WARNING: there is no transaction in progress
2026-08-12 06:56:59.092 UTC [941] WARNING: there is no transaction in progress
2026-08-12 06:56:59.109 UTC [941] WARNING: there is no transaction in progress
2026-08-12 06:56:59.121 UTC [941] WARNING: there is no transaction in progress
2026-08-12 06:56:59.129 UTC [941] WARNING: there is no transaction in progress
2026-08-12 06:56:59.138 UTC [941] WARNING: there is no transaction in progress
2026-08-12 06:56:59.147 UTC [941] WARNING: there is no transaction in progress
2026-08-12 06:57:00,016 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-12 06:57:00,016 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskG7Vh1a/build/debug/bin/postgresql.jar:/tmp/dist-test-taskG7Vh1a/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.19.52.84:57645/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-12 06:57:00,297 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-12 06:57:00,297 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskG7Vh1a/build/debug/bin/postgresql.jar:/tmp/dist-test-taskG7Vh1a/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.19.52.84:57645/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-12 06:57:00,573 [I] Table x_portal_user already exists in database 'ranger'
2026-08-12 06:57:00,574 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskG7Vh1a/build/debug/bin/postgresql.jar:/tmp/dist-test-taskG7Vh1a/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.19.52.84:57645/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-12 06:57:00,845 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-12 06:57:00,845 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskG7Vh1a/build/debug/bin/postgresql.jar:/tmp/dist-test-taskG7Vh1a/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.19.52.84:57645/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-12 06:57:01,121 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskG7Vh1a/build/debug/bin/postgresql.jar:/tmp/dist-test-taskG7Vh1a/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.19.52.84:57645/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-12 06:57:01,409 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskG7Vh1a/build/debug/bin/postgresql.jar:/tmp/dist-test-taskG7Vh1a/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.19.52.84:57645/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-65mx.c.gcp-upstream.internal';"
2026-08-12 06:57:01,684 [I] CORE_DB_SCHEMA import status has been updated
2026-08-12 06:57:01,684 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskG7Vh1a/build/debug/bin/postgresql.jar:/tmp/dist-test-taskG7Vh1a/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.19.52.84:57645/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-12 06:57:01,965 [I] DB_PATCHES have already been applied
I20260812 06:57:01.971657 19665 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517687384214-19665-0/minicluster-data/ranger-admin:/tmp/dist-test-taskG7Vh1a/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskG7Vh1a/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskG7Vh1a/build/debug/bin/postgresql.jar:/tmp/dist-test-taskG7Vh1a/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260812 06:57:01.971732 19665 mini_ranger.cc:193] Using host: 127.19.52.84
I20260812 06:57:01.976279 19665 mini_ranger.cc:227] Ranger admin URL: http://127.19.52.84:33283
Aug 12, 2026 6:57:02 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 12, 2026 6:57:02 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 12, 2026 6:57:02 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517687384214-19665-0/minicluster-data/ranger-admin/ews/webapp
Aug 12, 2026 6:57:02 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517687384214-19665-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 12, 2026 6:57:02 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517687384214-19665-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 12, 2026 6:57:02 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517687384214-19665-0/minicluster-data/ranger-admin/ews/webapp].
Aug 12, 2026 6:57:02 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260812 06:57:03.215517 19665 mini_ranger.cc:160] Time spent starting Ranger: real 9.640s user 0.000s sys 0.013s
Aug 12, 2026 6:57:24 AM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 12, 2026 6:57:25 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.PublicAPIsv2
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.UserREST
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.MetricsREST
Aug 12, 2026 6:57:25 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 12, 2026 6:57:25 AM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 12, 2026 6:57:25 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 12, 2026 6:57:25 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 12, 2026 6:57:25 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 12, 2026 6:57:25 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 12, 2026 6:57:25 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 12, 2026 6:57:25 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 12, 2026 6:57:25 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 12, 2026 6:57:25 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 12, 2026 6:57:25 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 12, 2026 6:57:25 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 12, 2026 6:57:25 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 12, 2026 6:57:25 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 12, 2026 6:57:25 AM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 12, 2026 6:57:26 AM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260812 06:57:26.682510 19665 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.19.52.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.19.52.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.19.52.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517687384214-19665-0/krb5kdc/rangerkms_127.19.52.84@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.19.52.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517687384214-19665-0/krb5kdc/rangerkms_127.19.52.84@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.19.52.84@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.19.52.84@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.19.52.84@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517687384214-19665-0/krb5kdc/HTTP_127.19.52.84@KRBTEST.COM.keytab.
Entry for principal HTTP/127.19.52.84@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517687384214-19665-0/krb5kdc/HTTP_127.19.52.84@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 12 06:57:26 dist-test-slave-65mx krb5kdc[707](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517846, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260812 06:57:26.783546 19665 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260812 06:57:26.794818 19665 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260812 06:57:27.864771 19665 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260812 06:57:27.864867 19665 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517687384214-19665-0/minicluster-data/ranger-kms
I20260812 06:57:27.864892 19665 mini_ranger_kms.cc:227] Using postgres at 127.19.52.84:57645
2026-08-12 06:57:28,049 [I] DB FLAVOR :POSTGRES
2026-08-12 06:57:28,049 [I] --------- Verifying Ranger DB connection ---------
2026-08-12 06:57:28,049 [I] Checking connection
2026-08-12 06:57:28,049 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskG7Vh1a/build/debug/bin/postgresql.jar:/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517687384214-19665-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.19.52.84:57645/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-12 06:57:28,316 [I] connection success
2026-08-12 06:57:28,316 [I] --------- Verifying Ranger DB tables ---------
2026-08-12 06:57:28,316 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-12 06:57:28,316 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskG7Vh1a/build/debug/bin/postgresql.jar:/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517687384214-19665-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.19.52.84:57645/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-12 06:57:28,605 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-12 06:57:28,606 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-12 06:57:28,606 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-12 06:57:28,606 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskG7Vh1a/build/debug/bin/postgresql.jar:/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517687384214-19665-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.19.52.84:57645/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517687384214-19665-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-12 06:57:28,903 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260812 06:57:29.446552 19665 mini_ranger_kms.cc:310] Created kms service
I20260812 06:57:29.678560 19665 mini_ranger_kms.cc:326] Created kudu user
I20260812 06:57:29.726509 19665 mini_ranger_kms.cc:343] Created rangerkms user
Aug 12 06:57:29 dist-test-slave-65mx krb5kdc[707](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517846, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.19.52.84@KRBTEST.COM
I20260812 06:57:30.177678 19665 mini_ranger_kms.cc:384] Added ranger policy
I20260812 06:57:30.177974 19665 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517687384214-19665-0/minicluster-data/ranger-kms:/tmp/dist-test-taskG7Vh1a/build/debug/bin/postgresql.jar:/tmp/dist-test-taskG7Vh1a/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskG7Vh1a/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260812 06:57:30.178010 19665 mini_ranger_kms.cc:240] Using host: 127.19.52.84
I20260812 06:57:30.183120 19665 mini_ranger_kms.cc:276] Ranger KMS PID: 1144
I20260812 06:57:30.183231 19665 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.19.52.84:44245
Aug 12, 2026 6:57:31 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 12, 2026 6:57:31 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 12, 2026 6:57:31 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517687384214-19665-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 12, 2026 6:57:31 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517687384214-19665-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 12, 2026 6:57:31 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517687384214-19665-0/minicluster-data/ranger-kms/ews/webapp].
Aug 12, 2026 6:57:31 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
Aug 12 06:57:32 dist-test-slave-65mx krb5kdc[707](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786517852, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.19.52.84@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 06:57:32.707131 19665 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 5.924s user 0.000s sys 0.167s
I20260812 06:57:32.707285 19665 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260812 06:57:32.707373 19665 mini_ranger_kms.cc:401] 127.19.52.84:44245/kms/v1/keys
Aug 12 06:57:32 dist-test-slave-65mx krb5kdc[707](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517852, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.19.52.84@KRBTEST.COM for HTTP/127.19.52.84@KRBTEST.COM
Aug 12 06:57:33 dist-test-slave-65mx krb5kdc[707](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517852, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.19.52.84@KRBTEST.COM for HTTP/127.19.52.84@KRBTEST.COM
Aug 12 06:57:35 dist-test-slave-65mx krb5kdc[707](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517855, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.19.52.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.52.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.52.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517687384214-19665-0/krb5kdc/kudu_127.19.52.126.keytab.
Entry for principal kudu/127.19.52.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517687384214-19665-0/krb5kdc/kudu_127.19.52.126.keytab.
I20260812 06:57:35.794222 19665 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu
/tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517687384214-19665-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517687384214-19665-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517687384214-19665-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517687384214-19665-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.52.126:44499
--webserver_interface=127.19.52.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517687384214-19665-0/krb5kdc/kudu_127.19.52.126.keytab
--principal=kudu/127.19.52.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.52.84:41511
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.19.52.126:44499
--ranger_config_path=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517687384214-19665-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.19.52.84:44245/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517687384214-19665-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517687384214-19665-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517687384214-19665-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 06:57:35.909437 1197 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:57:35.909734 1197 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:57:35.909775 1197 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 06:57:35.913972 1197 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 06:57:35.914047 1197 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 06:57:35.914067 1197 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 06:57:35.914083 1197 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 06:57:35.914124 1197 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 06:57:35.919775 1197 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.52.84:41511
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517687384214-19665-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517687384214-19665-0/minicluster-data/master-0/wal
--ranger_kms_url=127.19.52.84:44245/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.19.52.126:44499
--ranger_config_path=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517687384214-19665-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517687384214-19665-0/krb5kdc/kudu_127.19.52.126.keytab
--principal=kudu/127.19.52.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.52.126:44499
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517687384214-19665-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.19.52.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1197
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517687384214-19665-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 06:15:54 UTC on ccycloud.quasar-rnfzce.root.comops.site
build id 4581989
I20260812 06:57:35.921005 1197 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:57:35.922118 1197 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 06:57:35.931496 1206 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:57:35.931571 1197 server_base.cc:1034] running on GCE node
W20260812 06:57:35.931581 1208 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:57:35.931597 1204 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:57:35.932631 1197 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:57:35.933765 1197 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:57:35.934931 1197 hybrid_clock.cc:648] HybridClock initialized: now 1786517855934913 us; error 34 us; skew 500 ppm
Aug 12 06:57:35 dist-test-slave-65mx krb5kdc[707](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517855, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.52.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 06:57:35.942085 1197 init.cc:377] Logged in from keytab as kudu/127.19.52.126@KRBTEST.COM (short username kudu)
I20260812 06:57:35.943265 1197 webserver.cc:460] Webserver started at http://127.19.52.126:44505/ using document root <none> and password file <none>
I20260812 06:57:35.943840 1197 fs_manager.cc:359] Metadata directory not provided
I20260812 06:57:35.943899 1197 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:57:35.944182 1197 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 12 06:57:35 dist-test-slave-65mx krb5kdc[707](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517855, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.52.126@KRBTEST.COM for HTTP/127.19.52.84@KRBTEST.COM
I20260812 06:57:36.008381 1197 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517687384214-19665-0/minicluster-data/master-0/data/instance:
uuid: "cef00b9741644f17991e64877f67081d"
format_stamp: "Formatted at 2026-08-12 06:57:36 on dist-test-slave-65mx"
server_key: "5d8589a35b85e3a4a435308700674354"
server_key_iv: "cf6be4cc6dcf91b3ece4dc348f3b5ef1"
server_key_version: "kuduclusterkey@0"
I20260812 06:57:36.009104 1197 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517687384214-19665-0/minicluster-data/master-0/wal/instance:
uuid: "cef00b9741644f17991e64877f67081d"
format_stamp: "Formatted at 2026-08-12 06:57:36 on dist-test-slave-65mx"
server_key: "5d8589a35b85e3a4a435308700674354"
server_key_iv: "cf6be4cc6dcf91b3ece4dc348f3b5ef1"
server_key_version: "kuduclusterkey@0"
I20260812 06:57:36.013190 1197 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260812 06:57:36.032729 1216 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 06:57:36.034260 1197 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260812 06:57:36.034440 1197 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517687384214-19665-0/minicluster-data/master-0/data,/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517687384214-19665-0/minicluster-data/master-0/wal
uuid: "cef00b9741644f17991e64877f67081d"
format_stamp: "Formatted at 2026-08-12 06:57:36 on dist-test-slave-65mx"
server_key: "5d8589a35b85e3a4a435308700674354"
server_key_iv: "cf6be4cc6dcf91b3ece4dc348f3b5ef1"
server_key_version: "kuduclusterkey@0"
I20260812 06:57:36.034588 1197 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517687384214-19665-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517687384214-19665-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517687384214-19665-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:57:36.048225 1197 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 06:57:36.051590 1197 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 06:57:36.051813 1197 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:57:36.060700 1197 rpc_server.cc:307] RPC server started. Bound to: 127.19.52.126:44499
I20260812 06:57:36.060706 1269 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.52.126:44499 every 8 connection(s)
I20260812 06:57:36.061766 1197 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517687384214-19665-0/minicluster-data/master-0/data/info.pb
I20260812 06:57:36.064231 19665 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu as pid 1197
I20260812 06:57:36.064366 19665 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517687384214-19665-0/minicluster-data/master-0/wal/instance
I20260812 06:57:36.064746 19665 external_mini_cluster.cc:1428] Setting key 77afa38971afc98e8e1f1aad2a4d697e
I20260812 06:57:36.064855 1271 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 06:57:36.072340 1271 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cef00b9741644f17991e64877f67081d: Bootstrap starting.
I20260812 06:57:36.075260 1271 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P cef00b9741644f17991e64877f67081d: Neither blocks nor log segments found. Creating new log.
Aug 12 06:57:36 dist-test-slave-65mx krb5kdc[707](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517855, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.19.52.126@KRBTEST.COM
I20260812 06:57:36.076272 1271 log.cc:824] T 00000000000000000000000000000000 P cef00b9741644f17991e64877f67081d: Log is configured to *not* fsync() on all Append() calls
I20260812 06:57:36.078949 1271 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cef00b9741644f17991e64877f67081d: No bootstrap required, opened a new log
I20260812 06:57:36.082792 1271 raft_consensus.cc:348] T 00000000000000000000000000000000 P cef00b9741644f17991e64877f67081d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cef00b9741644f17991e64877f67081d" member_type: VOTER last_known_addr { host: "127.19.52.126" port: 44499 } }
I20260812 06:57:36.083061 1271 raft_consensus.cc:374] T 00000000000000000000000000000000 P cef00b9741644f17991e64877f67081d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 06:57:36.083151 1271 raft_consensus.cc:729] T 00000000000000000000000000000000 P cef00b9741644f17991e64877f67081d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cef00b9741644f17991e64877f67081d, State: Initialized, Role: FOLLOWER
I20260812 06:57:36.083367 1275 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:57:36.066374 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39428 (local address 127.19.52.126:44499)
0812 06:57:36.066856 (+ 482us) server_negotiation.cc:206] Beginning negotiation
0812 06:57:36.066875 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0812 06:57:36.066918 (+ 43us) server_negotiation.cc:407] Connection header received
0812 06:57:36.067821 (+ 903us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:57:36.067847 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:57:36.068277 (+ 430us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:57:36.068696 (+ 419us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:57:36.069673 (+ 977us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:57:36.071182 (+ 1509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:57:36.072457 (+ 1275us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:57:36.073182 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:57:36.073356 (+ 174us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:57:36.076694 (+ 3338us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:57:36.076701 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:57:36.076721 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:57:36.076756 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:57:36.079790 (+ 3034us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:57:36.080383 (+ 593us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:57:36.080390 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:57:36.080397 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:57:36.080570 (+ 173us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:57:36.080916 (+ 346us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:57:36.080919 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:57:36.080922 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:57:36.081469 (+ 547us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:57:36.081657 (+ 188us) server_negotiation.cc:1035] Waiting for connection context
0812 06:57:36.082019 (+ 362us) server_negotiation.cc:299] Negotiation successful
0812 06:57:36.082343 (+ 324us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":325,"thread_start_us":133,"threads_started":1}
I20260812 06:57:36.083771 1271 consensus_queue.cc:260] T 00000000000000000000000000000000 P cef00b9741644f17991e64877f67081d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cef00b9741644f17991e64877f67081d" member_type: VOTER last_known_addr { host: "127.19.52.126" port: 44499 } }
I20260812 06:57:36.083942 1271 raft_consensus.cc:388] T 00000000000000000000000000000000 P cef00b9741644f17991e64877f67081d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 06:57:36.084033 1271 raft_consensus.cc:482] T 00000000000000000000000000000000 P cef00b9741644f17991e64877f67081d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 06:57:36.084144 1271 raft_consensus.cc:3037] T 00000000000000000000000000000000 P cef00b9741644f17991e64877f67081d [term 0 FOLLOWER]: Advancing to term 1
I20260812 06:57:36.085412 1271 raft_consensus.cc:504] T 00000000000000000000000000000000 P cef00b9741644f17991e64877f67081d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cef00b9741644f17991e64877f67081d" member_type: VOTER last_known_addr { host: "127.19.52.126" port: 44499 } }
I20260812 06:57:36.085775 1271 leader_election.cc:302] T 00000000000000000000000000000000 P cef00b9741644f17991e64877f67081d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: cef00b9741644f17991e64877f67081d; no voters:
I20260812 06:57:36.086155 1271 leader_election.cc:288] T 00000000000000000000000000000000 P cef00b9741644f17991e64877f67081d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 06:57:36.086433 1277 raft_consensus.cc:2781] T 00000000000000000000000000000000 P cef00b9741644f17991e64877f67081d [term 1 FOLLOWER]: Leader election won for term 1
I20260812 06:57:36.086694 1277 raft_consensus.cc:686] T 00000000000000000000000000000000 P cef00b9741644f17991e64877f67081d [term 1 LEADER]: Becoming Leader. State: Replica: cef00b9741644f17991e64877f67081d, State: Running, Role: LEADER
I20260812 06:57:36.087075 1277 consensus_queue.cc:237] T 00000000000000000000000000000000 P cef00b9741644f17991e64877f67081d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cef00b9741644f17991e64877f67081d" member_type: VOTER last_known_addr { host: "127.19.52.126" port: 44499 } }
I20260812 06:57:36.087466 1271 sys_catalog.cc:564] T 00000000000000000000000000000000 P cef00b9741644f17991e64877f67081d [sys.catalog]: configured and running, proceeding with master startup.
I20260812 06:57:36.088861 1279 sys_catalog.cc:455] T 00000000000000000000000000000000 P cef00b9741644f17991e64877f67081d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "cef00b9741644f17991e64877f67081d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cef00b9741644f17991e64877f67081d" member_type: VOTER last_known_addr { host: "127.19.52.126" port: 44499 } } }
I20260812 06:57:36.088992 1279 sys_catalog.cc:458] T 00000000000000000000000000000000 P cef00b9741644f17991e64877f67081d [sys.catalog]: This master's current role is: LEADER
I20260812 06:57:36.089241 1279 sys_catalog.cc:455] T 00000000000000000000000000000000 P cef00b9741644f17991e64877f67081d [sys.catalog]: SysCatalogTable state changed. Reason: New leader cef00b9741644f17991e64877f67081d. Latest consensus state: current_term: 1 leader_uuid: "cef00b9741644f17991e64877f67081d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cef00b9741644f17991e64877f67081d" member_type: VOTER last_known_addr { host: "127.19.52.126" port: 44499 } } }
I20260812 06:57:36.089319 1279 sys_catalog.cc:458] T 00000000000000000000000000000000 P cef00b9741644f17991e64877f67081d [sys.catalog]: This master's current role is: LEADER
I20260812 06:57:36.089848 1271 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517687384214-19665-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260812 06:57:36.090418 1291 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 06:57:36.097594 1291 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 06:57:36.105034 1291 catalog_manager.cc:1347] Generated new cluster ID: 66dcedd6a2a04599896af3cfb080c331
I20260812 06:57:36.105221 1291 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 06:57:36.119524 1291 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 06:57:36.120859 1291 catalog_manager.cc:1504] Loading token signing keys...
I20260812 06:57:36.129431 1291 catalog_manager.cc:5868] T 00000000000000000000000000000000 P cef00b9741644f17991e64877f67081d: Generated new TSK 0
I20260812 06:57:36.130355 1291 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.19.52.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.52.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.52.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517687384214-19665-0/krb5kdc/kudu_127.19.52.65.keytab.
Entry for principal kudu/127.19.52.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517687384214-19665-0/krb5kdc/kudu_127.19.52.65.keytab.
I20260812 06:57:36.881428 19665 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu
/tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517687384214-19665-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517687384214-19665-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517687384214-19665-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517687384214-19665-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.19.52.84:44245/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.52.65:0
--local_ip_for_outbound_sockets=127.19.52.65
--webserver_interface=127.19.52.65
--webserver_port=0
--tserver_master_addrs=127.19.52.126:44499
--keytab_file=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517687384214-19665-0/krb5kdc/kudu_127.19.52.65.keytab
--principal=kudu/127.19.52.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.52.84:41511
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517687384214-19665-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517687384214-19665-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517687384214-19665-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 06:57:37.032877 1321 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:57:37.033205 1321 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:57:37.033324 1321 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 06:57:37.037642 1321 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 06:57:37.037740 1321 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 06:57:37.037832 1321 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.52.65
I20260812 06:57:37.043798 1321 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.52.84:41511
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517687384214-19665-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517687384214-19665-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.19.52.84:44245/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517687384214-19665-0/krb5kdc/kudu_127.19.52.65.keytab
--principal=kudu/127.19.52.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.52.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517687384214-19665-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.19.52.65
--webserver_port=0
--tserver_master_addrs=127.19.52.126:44499
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1321
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.52.65
--log_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517687384214-19665-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 06:15:54 UTC on ccycloud.quasar-rnfzce.root.comops.site
build id 4581989
I20260812 06:57:37.045195 1321 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:57:37.046307 1321 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260812 06:57:37.056753 1321 server_base.cc:1034] running on GCE node
W20260812 06:57:37.056996 1331 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:57:37.056671 1329 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:57:37.057272 1328 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:57:37.057778 1321 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:57:37.058526 1321 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:57:37.059758 1321 hybrid_clock.cc:648] HybridClock initialized: now 1786517857059722 us; error 49 us; skew 500 ppm
Aug 12 06:57:37 dist-test-slave-65mx krb5kdc[707](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517857, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.52.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 06:57:37.063628 1321 init.cc:377] Logged in from keytab as kudu/127.19.52.65@KRBTEST.COM (short username kudu)
I20260812 06:57:37.064769 1321 webserver.cc:460] Webserver started at http://127.19.52.65:41693/ using document root <none> and password file <none>
I20260812 06:57:37.065510 1321 fs_manager.cc:359] Metadata directory not provided
I20260812 06:57:37.065852 1321 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:57:37.066196 1321 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 12 06:57:37 dist-test-slave-65mx krb5kdc[707](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517857, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.52.65@KRBTEST.COM for HTTP/127.19.52.84@KRBTEST.COM
I20260812 06:57:37.084076 1321 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517687384214-19665-0/minicluster-data/ts-0/data/instance:
uuid: "97d4c4f7f749461f9119802f1c3a76f2"
format_stamp: "Formatted at 2026-08-12 06:57:37 on dist-test-slave-65mx"
server_key: "0452664001405a41688e6e54cc78982e"
server_key_iv: "2806cb39029b576938a9c69a43e31b9d"
server_key_version: "kuduclusterkey@0"
I20260812 06:57:37.084916 1321 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517687384214-19665-0/minicluster-data/ts-0/wal/instance:
uuid: "97d4c4f7f749461f9119802f1c3a76f2"
format_stamp: "Formatted at 2026-08-12 06:57:37 on dist-test-slave-65mx"
server_key: "0452664001405a41688e6e54cc78982e"
server_key_iv: "2806cb39029b576938a9c69a43e31b9d"
server_key_version: "kuduclusterkey@0"
I20260812 06:57:37.089545 1321 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260812 06:57:37.107959 1338 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 06:57:37.109834 1321 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260812 06:57:37.109983 1321 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517687384214-19665-0/minicluster-data/ts-0/data,/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517687384214-19665-0/minicluster-data/ts-0/wal
uuid: "97d4c4f7f749461f9119802f1c3a76f2"
format_stamp: "Formatted at 2026-08-12 06:57:37 on dist-test-slave-65mx"
server_key: "0452664001405a41688e6e54cc78982e"
server_key_iv: "2806cb39029b576938a9c69a43e31b9d"
server_key_version: "kuduclusterkey@0"
I20260812 06:57:37.110132 1321 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517687384214-19665-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517687384214-19665-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517687384214-19665-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:57:37.139525 1321 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 06:57:37.153213 1321 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 06:57:37.153539 1321 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:57:37.154381 1321 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 06:57:37.155558 1321 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 06:57:37.155635 1321 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 06:57:37.155694 1321 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 06:57:37.155751 1321 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.001s sys 0.000s
I20260812 06:57:37.170536 1321 rpc_server.cc:307] RPC server started. Bound to: 127.19.52.65:37993
I20260812 06:57:37.171968 1321 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517687384214-19665-0/minicluster-data/ts-0/data/info.pb
I20260812 06:57:37.173039 1467 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.52.65:37993 every 8 connection(s)
I20260812 06:57:37.177994 19665 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu as pid 1321
I20260812 06:57:37.178130 19665 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517687384214-19665-0/minicluster-data/ts-0/wal/instance
I20260812 06:57:37.178442 19665 external_mini_cluster.cc:1428] Setting key 2e784c6a2b6a706b42a4447ee652b204
Aug 12 06:57:37 dist-test-slave-65mx krb5kdc[707](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517857, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.52.65@KRBTEST.COM for kudu/127.19.52.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.19.52.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.52.66@KRBTEST.COM" created.
I20260812 06:57:37.199003 1471 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:57:37.177151 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.52.65:49185 (local address 127.19.52.126:44499)
0812 06:57:37.180197 (+ 3046us) server_negotiation.cc:206] Beginning negotiation
0812 06:57:37.180202 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 06:57:37.180216 (+ 14us) server_negotiation.cc:407] Connection header received
0812 06:57:37.180257 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:57:37.180260 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:57:37.180308 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:57:37.180392 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:57:37.182457 (+ 2065us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:57:37.183369 (+ 912us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:57:37.185471 (+ 2102us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:57:37.185863 (+ 392us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:57:37.185962 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:57:37.193622 (+ 7660us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:57:37.193630 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:57:37.193634 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:57:37.193673 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:57:37.195877 (+ 2204us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:57:37.196753 (+ 876us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:57:37.196756 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:57:37.196758 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:57:37.196860 (+ 102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:57:37.197376 (+ 516us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:57:37.197379 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:57:37.197381 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:57:37.197636 (+ 255us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:57:37.197728 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0812 06:57:37.198556 (+ 828us) server_negotiation.cc:299] Negotiation successful
0812 06:57:37.198715 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2957,"thread_start_us":91,"threads_started":1}
I20260812 06:57:37.200978 1470 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:57:37.175943 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.126:44499 (local address 127.19.52.65:49185)
0812 06:57:37.177655 (+ 1712us) negotiation.cc:107] Waiting for socket to connect
0812 06:57:37.177719 (+ 64us) client_negotiation.cc:174] Beginning negotiation
0812 06:57:37.178791 (+ 1072us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:57:37.181089 (+ 2298us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:57:37.181101 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:57:37.181497 (+ 396us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:57:37.182253 (+ 756us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:57:37.182277 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:57:37.183766 (+ 1489us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:57:37.183769 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:57:37.185307 (+ 1538us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:57:37.185320 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:57:37.186229 (+ 909us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:57:37.186234 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:57:37.186475 (+ 241us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:57:37.188053 (+ 1578us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 06:57:37.188070 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:57:37.189916 (+ 1846us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 06:57:37.196133 (+ 6217us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:57:37.196142 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:57:37.196163 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:57:37.196620 (+ 457us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:57:37.197016 (+ 396us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:57:37.197019 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:57:37.197021 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:57:37.197247 (+ 226us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:57:37.197804 (+ 557us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 06:57:37.197812 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 06:57:37.198240 (+ 428us) client_negotiation.cc:769] Sending connection context
0812 06:57:37.199337 (+ 1097us) client_negotiation.cc:240] Negotiation successful
0812 06:57:37.199749 (+ 412us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1467,"thread_start_us":147,"threads_started":1}
I20260812 06:57:37.202934 1468 heartbeater.cc:347] Connected to a master server at 127.19.52.126:44499
I20260812 06:57:37.203218 1468 heartbeater.cc:464] Registering TS with master...
I20260812 06:57:37.203771 1468 heartbeater.cc:511] Master 127.19.52.126:44499 requested a full tablet report, sending...
I20260812 06:57:37.206103 1233 ts_manager.cc:194] Registered new tserver with Master: 97d4c4f7f749461f9119802f1c3a76f2 (127.19.52.65:37993)
I20260812 06:57:37.208092 1233 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.52.65@KRBTEST.COM'} at 127.19.52.65:49185
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.52.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517687384214-19665-0/krb5kdc/kudu_127.19.52.66.keytab.
Entry for principal kudu/127.19.52.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517687384214-19665-0/krb5kdc/kudu_127.19.52.66.keytab.
I20260812 06:57:37.224663 19665 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu
/tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517687384214-19665-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517687384214-19665-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517687384214-19665-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517687384214-19665-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.19.52.84:44245/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.52.66:0
--local_ip_for_outbound_sockets=127.19.52.66
--webserver_interface=127.19.52.66
--webserver_port=0
--tserver_master_addrs=127.19.52.126:44499
--keytab_file=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517687384214-19665-0/krb5kdc/kudu_127.19.52.66.keytab
--principal=kudu/127.19.52.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.52.84:41511
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517687384214-19665-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517687384214-19665-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517687384214-19665-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 12 06:57:37 dist-test-slave-65mx krb5kdc[707](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786517857, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.52.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260812 06:57:37.397001 1474 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:57:37.397496 1474 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:57:37.397616 1474 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 06:57:37.404080 1474 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 06:57:37.404178 1474 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 06:57:37.404271 1474 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.52.66
I20260812 06:57:37.413285 1474 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.52.84:41511
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517687384214-19665-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517687384214-19665-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.19.52.84:44245/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517687384214-19665-0/krb5kdc/kudu_127.19.52.66.keytab
--principal=kudu/127.19.52.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.52.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517687384214-19665-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.19.52.66
--webserver_port=0
--tserver_master_addrs=127.19.52.126:44499
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1474
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.52.66
--log_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517687384214-19665-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 06:15:54 UTC on ccycloud.quasar-rnfzce.root.comops.site
build id 4581989
I20260812 06:57:37.415119 1474 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:57:37.416724 1474 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 06:57:37.431738 1481 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:57:37.431738 1484 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:57:37.432086 1474 server_base.cc:1034] running on GCE node
W20260812 06:57:37.431844 1480 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:57:37.432717 1474 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:57:37.433662 1474 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:57:37.434870 1474 hybrid_clock.cc:648] HybridClock initialized: now 1786517857434846 us; error 47 us; skew 500 ppm
Aug 12 06:57:37 dist-test-slave-65mx krb5kdc[707](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517857, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.52.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 06:57:37.438562 1474 init.cc:377] Logged in from keytab as kudu/127.19.52.66@KRBTEST.COM (short username kudu)
I20260812 06:57:37.439738 1474 webserver.cc:460] Webserver started at http://127.19.52.66:46369/ using document root <none> and password file <none>
I20260812 06:57:37.440366 1474 fs_manager.cc:359] Metadata directory not provided
I20260812 06:57:37.440469 1474 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:57:37.440742 1474 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 12 06:57:37 dist-test-slave-65mx krb5kdc[707](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517857, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.52.66@KRBTEST.COM for HTTP/127.19.52.84@KRBTEST.COM
I20260812 06:57:37.460232 1474 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517687384214-19665-0/minicluster-data/ts-1/data/instance:
uuid: "bef89111ca8e428ea48be43c25f58d46"
format_stamp: "Formatted at 2026-08-12 06:57:37 on dist-test-slave-65mx"
server_key: "40ae83cf48c76aa42726459857275e3c"
server_key_iv: "05fcb2088e2de9ab44040555f7dbc41a"
server_key_version: "kuduclusterkey@0"
I20260812 06:57:37.461145 1474 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517687384214-19665-0/minicluster-data/ts-1/wal/instance:
uuid: "bef89111ca8e428ea48be43c25f58d46"
format_stamp: "Formatted at 2026-08-12 06:57:37 on dist-test-slave-65mx"
server_key: "40ae83cf48c76aa42726459857275e3c"
server_key_iv: "05fcb2088e2de9ab44040555f7dbc41a"
server_key_version: "kuduclusterkey@0"
I20260812 06:57:37.473773 1474 fs_manager.cc:691] Time spent creating directory manager: real 0.012s user 0.004s sys 0.000s
I20260812 06:57:37.494238 1491 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 06:57:37.496075 1474 fs_manager.cc:725] Time spent opening block manager: real 0.007s user 0.002s sys 0.000s
I20260812 06:57:37.496237 1474 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517687384214-19665-0/minicluster-data/ts-1/data,/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517687384214-19665-0/minicluster-data/ts-1/wal
uuid: "bef89111ca8e428ea48be43c25f58d46"
format_stamp: "Formatted at 2026-08-12 06:57:37 on dist-test-slave-65mx"
server_key: "40ae83cf48c76aa42726459857275e3c"
server_key_iv: "05fcb2088e2de9ab44040555f7dbc41a"
server_key_version: "kuduclusterkey@0"
I20260812 06:57:37.496388 1474 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517687384214-19665-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517687384214-19665-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517687384214-19665-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:57:37.524629 1474 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 06:57:37.531038 1474 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 06:57:37.531344 1474 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:57:37.532186 1474 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 06:57:37.533903 1474 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 06:57:37.534091 1474 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.001s sys 0.000s
I20260812 06:57:37.534214 1474 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 06:57:37.534282 1474 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 06:57:37.546643 1474 rpc_server.cc:307] RPC server started. Bound to: 127.19.52.66:38375
I20260812 06:57:37.548054 1474 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517687384214-19665-0/minicluster-data/ts-1/data/info.pb
I20260812 06:57:37.549682 19665 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu as pid 1474
I20260812 06:57:37.549865 19665 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517687384214-19665-0/minicluster-data/ts-1/wal/instance
I20260812 06:57:37.550259 19665 external_mini_cluster.cc:1428] Setting key 6a84a9e562ed408e0d0c6fb27d0d7416
I20260812 06:57:37.561200 1625 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.52.66:38375 every 8 connection(s)
WARNING: no policy specified for kudu/127.19.52.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.52.67@KRBTEST.COM" created.
Aug 12 06:57:37 dist-test-slave-65mx krb5kdc[707](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517857, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.52.66@KRBTEST.COM for kudu/127.19.52.126@KRBTEST.COM
I20260812 06:57:37.590518 1471 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:57:37.558556 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.52.66:57177 (local address 127.19.52.126:44499)
0812 06:57:37.560229 (+ 1673us) server_negotiation.cc:206] Beginning negotiation
0812 06:57:37.560235 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 06:57:37.569142 (+ 8907us) server_negotiation.cc:407] Connection header received
0812 06:57:37.570566 (+ 1424us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:57:37.570571 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:57:37.570640 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:57:37.570704 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:57:37.572293 (+ 1589us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:57:37.573116 (+ 823us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:57:37.574955 (+ 1839us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:57:37.575380 (+ 425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:57:37.575483 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:57:37.585780 (+ 10297us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:57:37.585784 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:57:37.585787 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:57:37.585825 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:57:37.587610 (+ 1785us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:57:37.588440 (+ 830us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:57:37.588443 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:57:37.588445 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:57:37.588590 (+ 145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:57:37.589028 (+ 438us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:57:37.589033 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:57:37.589035 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:57:37.589293 (+ 258us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:57:37.589375 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0812 06:57:37.590197 (+ 822us) server_negotiation.cc:299] Negotiation successful
0812 06:57:37.590350 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1560}
I20260812 06:57:37.591529 1630 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:57:37.559087 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.126:44499 (local address 127.19.52.66:57177)
0812 06:57:37.568901 (+ 9814us) negotiation.cc:107] Waiting for socket to connect
0812 06:57:37.568996 (+ 95us) client_negotiation.cc:174] Beginning negotiation
0812 06:57:37.570217 (+ 1221us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:57:37.571169 (+ 952us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:57:37.571178 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:57:37.571481 (+ 303us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:57:37.572116 (+ 635us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:57:37.572140 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:57:37.573299 (+ 1159us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:57:37.573304 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:57:37.574790 (+ 1486us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:57:37.574806 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:57:37.575744 (+ 938us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:57:37.575752 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:57:37.576016 (+ 264us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:57:37.579433 (+ 3417us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 06:57:37.579462 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:57:37.584774 (+ 5312us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 06:57:37.587788 (+ 3014us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:57:37.587817 (+ 29us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:57:37.587845 (+ 28us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:57:37.588260 (+ 415us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:57:37.588685 (+ 425us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:57:37.588689 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:57:37.588691 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:57:37.588879 (+ 188us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:57:37.589488 (+ 609us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 06:57:37.589495 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 06:57:37.589881 (+ 386us) client_negotiation.cc:769] Sending connection context
0812 06:57:37.590155 (+ 274us) client_negotiation.cc:240] Negotiation successful
0812 06:57:37.590520 (+ 365us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":9470,"thread_start_us":155,"threads_started":1}
I20260812 06:57:37.593588 1627 heartbeater.cc:347] Connected to a master server at 127.19.52.126:44499
I20260812 06:57:37.593874 1627 heartbeater.cc:464] Registering TS with master...
I20260812 06:57:37.594427 1627 heartbeater.cc:511] Master 127.19.52.126:44499 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.52.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517687384214-19665-0/krb5kdc/kudu_127.19.52.67.keytab.
Entry for principal kudu/127.19.52.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517687384214-19665-0/krb5kdc/kudu_127.19.52.67.keytab.
I20260812 06:57:37.595636 1233 ts_manager.cc:194] Registered new tserver with Master: bef89111ca8e428ea48be43c25f58d46 (127.19.52.66:38375)
I20260812 06:57:37.596273 19665 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu
/tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517687384214-19665-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517687384214-19665-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517687384214-19665-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517687384214-19665-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.19.52.84:44245/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.52.67:0
--local_ip_for_outbound_sockets=127.19.52.67
--webserver_interface=127.19.52.67
--webserver_port=0
--tserver_master_addrs=127.19.52.126:44499
--keytab_file=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517687384214-19665-0/krb5kdc/kudu_127.19.52.67.keytab
--principal=kudu/127.19.52.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.52.84:41511
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517687384214-19665-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517687384214-19665-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517687384214-19665-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 06:57:37.596462 1233 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.52.66@KRBTEST.COM'} at 127.19.52.66:57177
W20260812 06:57:37.775318 1632 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:57:37.775732 1632 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:57:37.775816 1632 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 06:57:37.781956 1632 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 06:57:37.782063 1632 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 06:57:37.782168 1632 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.52.67
I20260812 06:57:37.791699 1632 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.52.84:41511
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517687384214-19665-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517687384214-19665-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.19.52.84:44245/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517687384214-19665-0/krb5kdc/kudu_127.19.52.67.keytab
--principal=kudu/127.19.52.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.52.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517687384214-19665-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.19.52.67
--webserver_port=0
--tserver_master_addrs=127.19.52.126:44499
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1632
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.52.67
--log_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517687384214-19665-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 06:15:54 UTC on ccycloud.quasar-rnfzce.root.comops.site
build id 4581989
I20260812 06:57:37.793542 1632 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:57:37.795316 1632 file_cache.cc:493] Constructed file cache file cache with capacity 419430
Aug 12 06:57:37 dist-test-slave-65mx krb5kdc[707](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517857, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.52.126@KRBTEST.COM for HTTP/127.19.52.84@KRBTEST.COM
W20260812 06:57:37.810490 1640 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:57:37.810544 1638 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:57:37.810978 1637 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:57:37.812690 1632 server_base.cc:1034] running on GCE node
I20260812 06:57:37.813261 1632 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:57:37.814193 1632 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:57:37.830490 1632 hybrid_clock.cc:648] HybridClock initialized: now 1786517857830351 us; error 141 us; skew 500 ppm
Aug 12 06:57:37 dist-test-slave-65mx krb5kdc[707](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517857, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.52.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 06:57:37.835500 1632 init.cc:377] Logged in from keytab as kudu/127.19.52.67@KRBTEST.COM (short username kudu)
I20260812 06:57:37.836859 1632 webserver.cc:460] Webserver started at http://127.19.52.67:42967/ using document root <none> and password file <none>
I20260812 06:57:37.837702 1632 fs_manager.cc:359] Metadata directory not provided
I20260812 06:57:37.837807 1632 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:57:37.838068 1632 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 12 06:57:37 dist-test-slave-65mx krb5kdc[707](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517857, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.52.67@KRBTEST.COM for HTTP/127.19.52.84@KRBTEST.COM
I20260812 06:57:37.855556 1632 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517687384214-19665-0/minicluster-data/ts-2/data/instance:
uuid: "e203f3ed23644e688a664be089c903f1"
format_stamp: "Formatted at 2026-08-12 06:57:37 on dist-test-slave-65mx"
server_key: "8736fedb5cfcf88ce95913e6560b9361"
server_key_iv: "aff3a8b64bf3821850530bc6acd72aff"
server_key_version: "kuduclusterkey@0"
I20260812 06:57:37.856375 1632 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517687384214-19665-0/minicluster-data/ts-2/wal/instance:
uuid: "e203f3ed23644e688a664be089c903f1"
format_stamp: "Formatted at 2026-08-12 06:57:37 on dist-test-slave-65mx"
server_key: "8736fedb5cfcf88ce95913e6560b9361"
server_key_iv: "aff3a8b64bf3821850530bc6acd72aff"
server_key_version: "kuduclusterkey@0"
I20260812 06:57:37.863778 1632 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.004s sys 0.001s
I20260812 06:57:37.913089 1648 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 06:57:37.915212 1632 fs_manager.cc:725] Time spent opening block manager: real 0.009s user 0.002s sys 0.000s
I20260812 06:57:37.915463 1632 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517687384214-19665-0/minicluster-data/ts-2/data,/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517687384214-19665-0/minicluster-data/ts-2/wal
uuid: "e203f3ed23644e688a664be089c903f1"
format_stamp: "Formatted at 2026-08-12 06:57:37 on dist-test-slave-65mx"
server_key: "8736fedb5cfcf88ce95913e6560b9361"
server_key_iv: "aff3a8b64bf3821850530bc6acd72aff"
server_key_version: "kuduclusterkey@0"
I20260812 06:57:37.915745 1632 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517687384214-19665-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517687384214-19665-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517687384214-19665-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:57:37.944108 1632 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 06:57:37.948369 1632 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 06:57:37.948697 1632 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:57:37.949581 1632 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 06:57:37.950870 1632 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 06:57:37.950979 1632 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 06:57:37.951092 1632 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 06:57:37.951154 1632 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 06:57:37.963817 1632 rpc_server.cc:307] RPC server started. Bound to: 127.19.52.67:37273
I20260812 06:57:37.963896 1775 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.52.67:37273 every 8 connection(s)
I20260812 06:57:37.964953 1632 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517687384214-19665-0/minicluster-data/ts-2/data/info.pb
I20260812 06:57:37.972767 19665 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu as pid 1632
I20260812 06:57:37.972934 19665 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517687384214-19665-0/minicluster-data/ts-2/wal/instance
I20260812 06:57:37.973275 19665 external_mini_cluster.cc:1428] Setting key ad1cd4f176d6d2a6c37339cc7c21b94b
Aug 12 06:57:37 dist-test-slave-65mx krb5kdc[707](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517857, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.52.67@KRBTEST.COM for kudu/127.19.52.126@KRBTEST.COM
I20260812 06:57:37.982139 1471 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:57:37.967379 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.52.67:54307 (local address 127.19.52.126:44499)
0812 06:57:37.967552 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0812 06:57:37.967558 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 06:57:37.968521 (+ 963us) server_negotiation.cc:407] Connection header received
0812 06:57:37.969705 (+ 1184us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:57:37.969708 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:57:37.969759 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:57:37.969870 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:57:37.971353 (+ 1483us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:57:37.971906 (+ 553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:57:37.973684 (+ 1778us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:57:37.973913 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:57:37.974066 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:57:37.977520 (+ 3454us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:57:37.977524 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:57:37.977527 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:57:37.977558 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:57:37.979399 (+ 1841us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:57:37.980057 (+ 658us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:57:37.980061 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:57:37.980063 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:57:37.980160 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:57:37.980844 (+ 684us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:57:37.980848 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:57:37.980850 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:57:37.981116 (+ 266us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:57:37.981224 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0812 06:57:37.981813 (+ 589us) server_negotiation.cc:299] Negotiation successful
0812 06:57:37.981952 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260812 06:57:37.983040 1780 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:57:37.967782 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.126:44499 (local address 127.19.52.67:54307)
0812 06:57:37.968321 (+ 539us) negotiation.cc:107] Waiting for socket to connect
0812 06:57:37.968380 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0812 06:57:37.969459 (+ 1079us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:57:37.970050 (+ 591us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:57:37.970060 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:57:37.970456 (+ 396us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:57:37.971163 (+ 707us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:57:37.971187 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:57:37.972052 (+ 865us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:57:37.972055 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:57:37.973509 (+ 1454us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:57:37.973521 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:57:37.974122 (+ 601us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:57:37.974127 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:57:37.974350 (+ 223us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:57:37.975327 (+ 977us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 06:57:37.975350 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:57:37.977326 (+ 1976us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 06:57:37.979557 (+ 2231us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:57:37.979562 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:57:37.979578 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:57:37.979912 (+ 334us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:57:37.980289 (+ 377us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:57:37.980292 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:57:37.980294 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:57:37.980474 (+ 180us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:57:37.981230 (+ 756us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 06:57:37.981237 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 06:57:37.981510 (+ 273us) client_negotiation.cc:769] Sending connection context
0812 06:57:37.981703 (+ 193us) client_negotiation.cc:240] Negotiation successful
0812 06:57:37.981993 (+ 290us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":317,"thread_start_us":128,"threads_started":1}
I20260812 06:57:37.985010 1776 heartbeater.cc:347] Connected to a master server at 127.19.52.126:44499
I20260812 06:57:37.985297 1776 heartbeater.cc:464] Registering TS with master...
I20260812 06:57:37.985929 1776 heartbeater.cc:511] Master 127.19.52.126:44499 requested a full tablet report, sending...
I20260812 06:57:37.987107 1233 ts_manager.cc:194] Registered new tserver with Master: e203f3ed23644e688a664be089c903f1 (127.19.52.67:37273)
I20260812 06:57:37.987810 19665 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 06:57:37.987898 1233 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.52.67@KRBTEST.COM'} at 127.19.52.67:54307
I20260812 06:57:37.988757 19665 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu with pid 1321
I20260812 06:57:38.005184 19665 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu with pid 1474
I20260812 06:57:38.014685 19665 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu with pid 1632
I20260812 06:57:38.022239 19665 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu with pid 1197
I20260812 06:57:38.032040 19665 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu
/tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517687384214-19665-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517687384214-19665-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517687384214-19665-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517687384214-19665-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.52.126:44499
--webserver_interface=127.19.52.126
--webserver_port=44505
--keytab_file=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517687384214-19665-0/krb5kdc/kudu_127.19.52.126.keytab
--principal=kudu/127.19.52.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.52.84:41511
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.19.52.126:44499
--ranger_config_path=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517687384214-19665-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.19.52.84:44245/kms
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.19.52.84:44245/kms with env {KRB5CCNAME=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517687384214-19665-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517687384214-19665-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517687384214-19665-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 06:57:38.154767 1783 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:57:38.155098 1783 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:57:38.155141 1783 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 06:57:38.159597 1783 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 06:57:38.159678 1783 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 06:57:38.159698 1783 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 06:57:38.159715 1783 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 06:57:38.159731 1783 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 06:57:38.165697 1783 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.52.84:41511
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517687384214-19665-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517687384214-19665-0/minicluster-data/master-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.19.52.84:44245/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.19.52.126:44499
--ranger_config_path=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517687384214-19665-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517687384214-19665-0/krb5kdc/kudu_127.19.52.126.keytab
--principal=kudu/127.19.52.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.52.126:44499
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517687384214-19665-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.19.52.126
--webserver_port=44505
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1783
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517687384214-19665-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 06:15:54 UTC on ccycloud.quasar-rnfzce.root.comops.site
build id 4581989
I20260812 06:57:38.166893 1783 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:57:38.167979 1783 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 06:57:38.177439 1791 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:57:38.177423 1789 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:57:38.177503 1783 server_base.cc:1034] running on GCE node
W20260812 06:57:38.177429 1788 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:57:38.178341 1783 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:57:38.179574 1783 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:57:38.180788 1783 hybrid_clock.cc:648] HybridClock initialized: now 1786517858180764 us; error 41 us; skew 500 ppm
Aug 12 06:57:38 dist-test-slave-65mx krb5kdc[707](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517858, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.52.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 06:57:38.184286 1783 init.cc:377] Logged in from keytab as kudu/127.19.52.126@KRBTEST.COM (short username kudu)
I20260812 06:57:38.185432 1783 webserver.cc:460] Webserver started at http://127.19.52.126:44505/ using document root <none> and password file <none>
I20260812 06:57:38.186031 1783 fs_manager.cc:359] Metadata directory not provided
I20260812 06:57:38.186113 1783 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 12 06:57:38 dist-test-slave-65mx krb5kdc[707](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517858, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.52.126@KRBTEST.COM for HTTP/127.19.52.84@KRBTEST.COM
I20260812 06:57:38.229380 1783 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.000s
I20260812 06:57:38.232149 1799 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 06:57:38.233716 1783 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 06:57:38.233876 1783 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517687384214-19665-0/minicluster-data/master-0/data,/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517687384214-19665-0/minicluster-data/master-0/wal
uuid: "cef00b9741644f17991e64877f67081d"
format_stamp: "Formatted at 2026-08-12 06:57:36 on dist-test-slave-65mx"
server_key: "5d8589a35b85e3a4a435308700674354"
server_key_iv: "cf6be4cc6dcf91b3ece4dc348f3b5ef1"
server_key_version: "kuduclusterkey@0"
I20260812 06:57:38.234512 1783 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517687384214-19665-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517687384214-19665-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517687384214-19665-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:57:38.281821 1783 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 06:57:38.285094 1783 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 06:57:38.285339 1783 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:57:38.292809 1783 rpc_server.cc:307] RPC server started. Bound to: 127.19.52.126:44499
I20260812 06:57:38.292838 1857 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.52.126:44499 every 8 connection(s)
I20260812 06:57:38.293993 1783 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517687384214-19665-0/minicluster-data/master-0/data/info.pb
I20260812 06:57:38.298408 1858 sys_catalog.cc:263] Verifying existing consensus state
I20260812 06:57:38.300118 1858 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cef00b9741644f17991e64877f67081d: Bootstrap starting.
I20260812 06:57:38.302078 19665 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu as pid 1783
I20260812 06:57:38.302564 19665 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu
/tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517687384214-19665-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517687384214-19665-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517687384214-19665-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517687384214-19665-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.19.52.84:44245/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.52.65:37993
--local_ip_for_outbound_sockets=127.19.52.65
--tserver_master_addrs=127.19.52.126:44499
--webserver_port=41693
--webserver_interface=127.19.52.65
--keytab_file=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517687384214-19665-0/krb5kdc/kudu_127.19.52.65.keytab
--principal=kudu/127.19.52.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.52.84:41511
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.19.52.84:44245/kms with env {KRB5CCNAME=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517687384214-19665-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517687384214-19665-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517687384214-19665-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 06:57:38.318181 1858 log.cc:824] T 00000000000000000000000000000000 P cef00b9741644f17991e64877f67081d: Log is configured to *not* fsync() on all Append() calls
I20260812 06:57:38.325786 1858 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cef00b9741644f17991e64877f67081d: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260812 06:57:38.326236 1858 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cef00b9741644f17991e64877f67081d: Bootstrap complete.
I20260812 06:57:38.330763 1858 raft_consensus.cc:348] T 00000000000000000000000000000000 P cef00b9741644f17991e64877f67081d [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cef00b9741644f17991e64877f67081d" member_type: VOTER last_known_addr { host: "127.19.52.126" port: 44499 } }
I20260812 06:57:38.331243 1858 raft_consensus.cc:729] T 00000000000000000000000000000000 P cef00b9741644f17991e64877f67081d [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: cef00b9741644f17991e64877f67081d, State: Initialized, Role: FOLLOWER
I20260812 06:57:38.331809 1858 consensus_queue.cc:260] T 00000000000000000000000000000000 P cef00b9741644f17991e64877f67081d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cef00b9741644f17991e64877f67081d" member_type: VOTER last_known_addr { host: "127.19.52.126" port: 44499 } }
I20260812 06:57:38.331964 1858 raft_consensus.cc:388] T 00000000000000000000000000000000 P cef00b9741644f17991e64877f67081d [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 06:57:38.332129 1858 raft_consensus.cc:482] T 00000000000000000000000000000000 P cef00b9741644f17991e64877f67081d [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 06:57:38.332252 1858 raft_consensus.cc:3037] T 00000000000000000000000000000000 P cef00b9741644f17991e64877f67081d [term 1 FOLLOWER]: Advancing to term 2
I20260812 06:57:38.334388 1858 raft_consensus.cc:504] T 00000000000000000000000000000000 P cef00b9741644f17991e64877f67081d [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cef00b9741644f17991e64877f67081d" member_type: VOTER last_known_addr { host: "127.19.52.126" port: 44499 } }
I20260812 06:57:38.334755 1858 leader_election.cc:302] T 00000000000000000000000000000000 P cef00b9741644f17991e64877f67081d [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: cef00b9741644f17991e64877f67081d; no voters:
I20260812 06:57:38.335180 1858 leader_election.cc:288] T 00000000000000000000000000000000 P cef00b9741644f17991e64877f67081d [CANDIDATE]: Term 2 election: Requested vote from peers
I20260812 06:57:38.335285 1863 raft_consensus.cc:2781] T 00000000000000000000000000000000 P cef00b9741644f17991e64877f67081d [term 2 FOLLOWER]: Leader election won for term 2
I20260812 06:57:38.335513 1863 raft_consensus.cc:686] T 00000000000000000000000000000000 P cef00b9741644f17991e64877f67081d [term 2 LEADER]: Becoming Leader. State: Replica: cef00b9741644f17991e64877f67081d, State: Running, Role: LEADER
I20260812 06:57:38.335824 1863 consensus_queue.cc:237] T 00000000000000000000000000000000 P cef00b9741644f17991e64877f67081d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cef00b9741644f17991e64877f67081d" member_type: VOTER last_known_addr { host: "127.19.52.126" port: 44499 } }
I20260812 06:57:38.336566 1858 sys_catalog.cc:564] T 00000000000000000000000000000000 P cef00b9741644f17991e64877f67081d [sys.catalog]: configured and running, proceeding with master startup.
I20260812 06:57:38.337539 1865 sys_catalog.cc:455] T 00000000000000000000000000000000 P cef00b9741644f17991e64877f67081d [sys.catalog]: SysCatalogTable state changed. Reason: New leader cef00b9741644f17991e64877f67081d. Latest consensus state: current_term: 2 leader_uuid: "cef00b9741644f17991e64877f67081d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cef00b9741644f17991e64877f67081d" member_type: VOTER last_known_addr { host: "127.19.52.126" port: 44499 } } }
I20260812 06:57:38.337595 1864 sys_catalog.cc:455] T 00000000000000000000000000000000 P cef00b9741644f17991e64877f67081d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "cef00b9741644f17991e64877f67081d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cef00b9741644f17991e64877f67081d" member_type: VOTER last_known_addr { host: "127.19.52.126" port: 44499 } } }
I20260812 06:57:38.337733 1864 sys_catalog.cc:458] T 00000000000000000000000000000000 P cef00b9741644f17991e64877f67081d [sys.catalog]: This master's current role is: LEADER
I20260812 06:57:38.338176 1872 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 06:57:38.337735 1865 sys_catalog.cc:458] T 00000000000000000000000000000000 P cef00b9741644f17991e64877f67081d [sys.catalog]: This master's current role is: LEADER
I20260812 06:57:38.341814 1872 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 06:57:38.342401 1858 ranger_client.cc:318] Using overwritten properties file: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517687384214-19665-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260812 06:57:38.342475 1872 catalog_manager.cc:1259] Loaded cluster ID: 66dcedd6a2a04599896af3cfb080c331
I20260812 06:57:38.342542 1872 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 06:57:38.344986 1872 catalog_manager.cc:1504] Loading token signing keys...
I20260812 06:57:38.345690 1872 catalog_manager.cc:5879] T 00000000000000000000000000000000 P cef00b9741644f17991e64877f67081d: Loaded TSK: 0
I20260812 06:57:38.346930 1872 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260812 06:57:38.444711 1860 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:57:38.445026 1860 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:57:38.445112 1860 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 06:57:38.449551 1860 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 06:57:38.449647 1860 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 06:57:38.449753 1860 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.52.65
I20260812 06:57:38.456812 1860 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.52.84:41511
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517687384214-19665-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517687384214-19665-0/minicluster-data/ts-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.19.52.84:44245/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517687384214-19665-0/krb5kdc/kudu_127.19.52.65.keytab
--principal=kudu/127.19.52.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.52.65:37993
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517687384214-19665-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.19.52.65
--webserver_port=41693
--tserver_master_addrs=127.19.52.126:44499
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1860
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.52.65
--log_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517687384214-19665-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 06:15:54 UTC on ccycloud.quasar-rnfzce.root.comops.site
build id 4581989
I20260812 06:57:38.458263 1860 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:57:38.459662 1860 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 06:57:38.472687 1902 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:57:38.473272 1860 server_base.cc:1034] running on GCE node
W20260812 06:57:38.472776 1901 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:57:38.473024 1905 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:57:38.473984 1860 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:57:38.474853 1860 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:57:38.476078 1860 hybrid_clock.cc:648] HybridClock initialized: now 1786517858476052 us; error 42 us; skew 500 ppm
Aug 12 06:57:38 dist-test-slave-65mx krb5kdc[707](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517858, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.52.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 06:57:38.479933 1860 init.cc:377] Logged in from keytab as kudu/127.19.52.65@KRBTEST.COM (short username kudu)
I20260812 06:57:38.481232 1860 webserver.cc:460] Webserver started at http://127.19.52.65:41693/ using document root <none> and password file <none>
I20260812 06:57:38.481979 1860 fs_manager.cc:359] Metadata directory not provided
I20260812 06:57:38.482102 1860 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 12 06:57:38 dist-test-slave-65mx krb5kdc[707](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517858, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.52.65@KRBTEST.COM for HTTP/127.19.52.84@KRBTEST.COM
I20260812 06:57:38.506565 1860 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.000s
I20260812 06:57:38.509949 1913 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 06:57:38.511700 1860 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260812 06:57:38.511822 1860 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517687384214-19665-0/minicluster-data/ts-0/data,/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517687384214-19665-0/minicluster-data/ts-0/wal
uuid: "97d4c4f7f749461f9119802f1c3a76f2"
format_stamp: "Formatted at 2026-08-12 06:57:37 on dist-test-slave-65mx"
server_key: "0452664001405a41688e6e54cc78982e"
server_key_iv: "2806cb39029b576938a9c69a43e31b9d"
server_key_version: "kuduclusterkey@0"
I20260812 06:57:38.512617 1860 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517687384214-19665-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517687384214-19665-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517687384214-19665-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:57:38.542090 1860 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 06:57:38.545879 1860 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 06:57:38.546150 1860 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:57:38.547034 1860 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 06:57:38.548089 1860 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 06:57:38.548156 1860 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 06:57:38.548233 1860 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 06:57:38.548264 1860 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 06:57:38.560176 1860 rpc_server.cc:307] RPC server started. Bound to: 127.19.52.65:37993
I20260812 06:57:38.560736 2056 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.52.65:37993 every 8 connection(s)
I20260812 06:57:38.561728 1860 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517687384214-19665-0/minicluster-data/ts-0/data/info.pb
I20260812 06:57:38.562966 19665 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu as pid 1860
I20260812 06:57:38.563506 19665 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu
/tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517687384214-19665-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517687384214-19665-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517687384214-19665-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517687384214-19665-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.19.52.84:44245/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.52.66:38375
--local_ip_for_outbound_sockets=127.19.52.66
--tserver_master_addrs=127.19.52.126:44499
--webserver_port=46369
--webserver_interface=127.19.52.66
--keytab_file=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517687384214-19665-0/krb5kdc/kudu_127.19.52.66.keytab
--principal=kudu/127.19.52.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.52.84:41511
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.19.52.84:44245/kms with env {KRB5CCNAME=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517687384214-19665-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517687384214-19665-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517687384214-19665-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 12 06:57:38 dist-test-slave-65mx krb5kdc[707](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517858, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.52.65@KRBTEST.COM for kudu/127.19.52.126@KRBTEST.COM
I20260812 06:57:38.596051 2060 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:57:38.567004 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.126:44499 (local address 127.19.52.65:34625)
0812 06:57:38.572992 (+ 5988us) negotiation.cc:107] Waiting for socket to connect
0812 06:57:38.573083 (+ 91us) client_negotiation.cc:174] Beginning negotiation
0812 06:57:38.574291 (+ 1208us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:57:38.576450 (+ 2159us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:57:38.576462 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:57:38.576901 (+ 439us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:57:38.577705 (+ 804us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:57:38.577727 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:57:38.580281 (+ 2554us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:57:38.580286 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:57:38.581781 (+ 1495us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:57:38.581796 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:57:38.582986 (+ 1190us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:57:38.582991 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:57:38.583214 (+ 223us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:57:38.584008 (+ 794us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 06:57:38.584031 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:57:38.589085 (+ 5054us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 06:57:38.592046 (+ 2961us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:57:38.592055 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:57:38.592073 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:57:38.592485 (+ 412us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:57:38.593006 (+ 521us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:57:38.593010 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:57:38.593012 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:57:38.593242 (+ 230us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:57:38.594080 (+ 838us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 06:57:38.594087 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 06:57:38.594397 (+ 310us) client_negotiation.cc:769] Sending connection context
0812 06:57:38.594593 (+ 196us) client_negotiation.cc:240] Negotiation successful
0812 06:57:38.594951 (+ 358us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5670,"thread_start_us":157,"threads_started":1}
I20260812 06:57:38.598418 2061 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:57:38.568685 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.52.65:34625 (local address 127.19.52.126:44499)
0812 06:57:38.574937 (+ 6252us) server_negotiation.cc:206] Beginning negotiation
0812 06:57:38.574966 (+ 29us) server_negotiation.cc:399] Waiting for connection header
0812 06:57:38.575045 (+ 79us) server_negotiation.cc:407] Connection header received
0812 06:57:38.575381 (+ 336us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:57:38.575396 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:57:38.575920 (+ 524us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:57:38.577985 (+ 2065us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:57:38.578329 (+ 344us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:57:38.580087 (+ 1758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:57:38.581968 (+ 1881us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:57:38.582766 (+ 798us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:57:38.583723 (+ 957us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:57:38.589419 (+ 5696us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:57:38.589428 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:57:38.589448 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:57:38.589500 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:57:38.591846 (+ 2346us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:57:38.592701 (+ 855us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:57:38.592708 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:57:38.592714 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:57:38.592871 (+ 157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:57:38.593379 (+ 508us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:57:38.593383 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:57:38.593385 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:57:38.593919 (+ 534us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:57:38.596861 (+ 2942us) server_negotiation.cc:1035] Waiting for connection context
0812 06:57:38.597078 (+ 217us) server_negotiation.cc:299] Negotiation successful
0812 06:57:38.597422 (+ 344us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":6041,"thread_start_us":130,"threads_started":1}
I20260812 06:57:38.601337 2057 heartbeater.cc:347] Connected to a master server at 127.19.52.126:44499
I20260812 06:57:38.601631 2057 heartbeater.cc:464] Registering TS with master...
I20260812 06:57:38.602278 2057 heartbeater.cc:511] Master 127.19.52.126:44499 requested a full tablet report, sending...
W20260812 06:57:38.605279 2057 heartbeater.cc:644] Failed to heartbeat to 127.19.52.126:44499 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260812 06:57:38.605425 2057 heartbeater.cc:464] Registering TS with master...
I20260812 06:57:38.605644 2057 heartbeater.cc:511] Master 127.19.52.126:44499 requested a full tablet report, sending...
I20260812 06:57:38.606810 2057 heartbeater.cc:464] Registering TS with master...
I20260812 06:57:38.607017 2057 heartbeater.cc:511] Master 127.19.52.126:44499 requested a full tablet report, sending...
W20260812 06:57:38.608007 2057 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260812 06:57:38.725382 2059 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:57:38.725827 2059 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:57:38.725917 2059 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 06:57:38.731211 2059 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 06:57:38.731287 2059 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 06:57:38.731380 2059 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.52.66
I20260812 06:57:38.737979 2059 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.52.84:41511
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517687384214-19665-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517687384214-19665-0/minicluster-data/ts-1/wal
--ranger_kms_url=invalid.host:1234/kms,127.19.52.84:44245/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517687384214-19665-0/krb5kdc/kudu_127.19.52.66.keytab
--principal=kudu/127.19.52.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.52.66:38375
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517687384214-19665-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.19.52.66
--webserver_port=46369
--tserver_master_addrs=127.19.52.126:44499
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2059
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.52.66
--log_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517687384214-19665-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 06:15:54 UTC on ccycloud.quasar-rnfzce.root.comops.site
build id 4581989
I20260812 06:57:38.739377 2059 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:57:38.740788 2059 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 06:57:38.754107 2067 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:57:38.754257 2068 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:57:38.754117 2070 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:57:38.754637 2059 server_base.cc:1034] running on GCE node
I20260812 06:57:38.755152 2059 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:57:38.755921 2059 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:57:38.757102 2059 hybrid_clock.cc:648] HybridClock initialized: now 1786517858757067 us; error 45 us; skew 500 ppm
Aug 12 06:57:38 dist-test-slave-65mx krb5kdc[707](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517858, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.52.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 06:57:38.760061 2059 init.cc:377] Logged in from keytab as kudu/127.19.52.66@KRBTEST.COM (short username kudu)
I20260812 06:57:38.761243 2059 webserver.cc:460] Webserver started at http://127.19.52.66:46369/ using document root <none> and password file <none>
I20260812 06:57:38.761865 2059 fs_manager.cc:359] Metadata directory not provided
I20260812 06:57:38.761991 2059 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 12 06:57:38 dist-test-slave-65mx krb5kdc[707](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517858, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.52.66@KRBTEST.COM for HTTP/127.19.52.84@KRBTEST.COM
I20260812 06:57:38.784216 2059 fs_manager.cc:709] Time spent opening directory manager: real 0.005s user 0.003s sys 0.000s
I20260812 06:57:38.787701 2078 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 06:57:38.789467 2059 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.000s
I20260812 06:57:38.789610 2059 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517687384214-19665-0/minicluster-data/ts-1/data,/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517687384214-19665-0/minicluster-data/ts-1/wal
uuid: "bef89111ca8e428ea48be43c25f58d46"
format_stamp: "Formatted at 2026-08-12 06:57:37 on dist-test-slave-65mx"
server_key: "40ae83cf48c76aa42726459857275e3c"
server_key_iv: "05fcb2088e2de9ab44040555f7dbc41a"
server_key_version: "kuduclusterkey@0"
I20260812 06:57:38.790359 2059 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517687384214-19665-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517687384214-19665-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517687384214-19665-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:57:38.821333 2059 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 06:57:38.825490 2059 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 06:57:38.825740 2059 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:57:38.826529 2059 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 06:57:38.827647 2059 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 06:57:38.827718 2059 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 06:57:38.827800 2059 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 06:57:38.827834 2059 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 06:57:38.841077 2059 rpc_server.cc:307] RPC server started. Bound to: 127.19.52.66:38375
I20260812 06:57:38.841096 2211 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.52.66:38375 every 8 connection(s)
I20260812 06:57:38.842264 2059 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517687384214-19665-0/minicluster-data/ts-1/data/info.pb
I20260812 06:57:38.844349 19665 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu as pid 2059
I20260812 06:57:38.844944 19665 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu
/tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517687384214-19665-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517687384214-19665-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517687384214-19665-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517687384214-19665-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.19.52.84:44245/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.52.67:37273
--local_ip_for_outbound_sockets=127.19.52.67
--tserver_master_addrs=127.19.52.126:44499
--webserver_port=42967
--webserver_interface=127.19.52.67
--keytab_file=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517687384214-19665-0/krb5kdc/kudu_127.19.52.67.keytab
--principal=kudu/127.19.52.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.52.84:41511
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.19.52.84:44245/kms with env {KRB5CCNAME=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517687384214-19665-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517687384214-19665-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517687384214-19665-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 12 06:57:38 dist-test-slave-65mx krb5kdc[707](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517858, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.52.66@KRBTEST.COM for kudu/127.19.52.126@KRBTEST.COM
I20260812 06:57:38.870679 2061 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:57:38.844787 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.52.66:42359 (local address 127.19.52.126:44499)
0812 06:57:38.844941 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0812 06:57:38.844946 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 06:57:38.855490 (+ 10544us) server_negotiation.cc:407] Connection header received
0812 06:57:38.856967 (+ 1477us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:57:38.856972 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:57:38.857039 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:57:38.857102 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:57:38.858852 (+ 1750us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:57:38.859703 (+ 851us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:57:38.861555 (+ 1852us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:57:38.861924 (+ 369us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:57:38.862015 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:57:38.865762 (+ 3747us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:57:38.865767 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:57:38.865770 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:57:38.865806 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:57:38.867716 (+ 1910us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:57:38.868722 (+ 1006us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:57:38.868726 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:57:38.868728 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:57:38.868827 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:57:38.869244 (+ 417us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:57:38.869248 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:57:38.869250 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:57:38.869485 (+ 235us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:57:38.869580 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0812 06:57:38.870328 (+ 748us) server_negotiation.cc:299] Negotiation successful
0812 06:57:38.870483 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260812 06:57:38.873158 2214 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:57:38.845758 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.126:44499 (local address 127.19.52.66:42359)
0812 06:57:38.855232 (+ 9474us) negotiation.cc:107] Waiting for socket to connect
0812 06:57:38.855334 (+ 102us) client_negotiation.cc:174] Beginning negotiation
0812 06:57:38.856673 (+ 1339us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:57:38.857557 (+ 884us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:57:38.857567 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:57:38.857954 (+ 387us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:57:38.858661 (+ 707us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:57:38.858682 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:57:38.859889 (+ 1207us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:57:38.859892 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:57:38.861400 (+ 1508us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:57:38.861412 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:57:38.862269 (+ 857us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:57:38.862274 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:57:38.862499 (+ 225us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:57:38.863281 (+ 782us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 06:57:38.863304 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:57:38.865264 (+ 1960us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 06:57:38.867879 (+ 2615us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:57:38.867886 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:57:38.867906 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:57:38.868591 (+ 685us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:57:38.868939 (+ 348us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:57:38.868942 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:57:38.868944 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:57:38.869135 (+ 191us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:57:38.869649 (+ 514us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 06:57:38.869656 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 06:57:38.870011 (+ 355us) client_negotiation.cc:769] Sending connection context
0812 06:57:38.871199 (+ 1188us) client_negotiation.cc:240] Negotiation successful
0812 06:57:38.871583 (+ 384us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":9047,"thread_start_us":188,"threads_started":1}
I20260812 06:57:38.874974 2212 heartbeater.cc:347] Connected to a master server at 127.19.52.126:44499
I20260812 06:57:38.875226 2212 heartbeater.cc:464] Registering TS with master...
I20260812 06:57:38.875805 2212 heartbeater.cc:511] Master 127.19.52.126:44499 requested a full tablet report, sending...
W20260812 06:57:38.877668 2212 heartbeater.cc:644] Failed to heartbeat to 127.19.52.126:44499 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260812 06:57:38.877794 2212 heartbeater.cc:464] Registering TS with master...
I20260812 06:57:38.877970 2212 heartbeater.cc:511] Master 127.19.52.126:44499 requested a full tablet report, sending...
I20260812 06:57:38.878885 2212 heartbeater.cc:464] Registering TS with master...
I20260812 06:57:38.879060 2212 heartbeater.cc:511] Master 127.19.52.126:44499 requested a full tablet report, sending...
W20260812 06:57:38.879875 2212 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260812 06:57:39.018546 2215 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:57:39.018982 2215 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:57:39.019076 2215 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 06:57:39.025311 2215 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 06:57:39.025429 2215 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 06:57:39.025543 2215 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.52.67
I20260812 06:57:39.032146 2215 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.52.84:41511
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517687384214-19665-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517687384214-19665-0/minicluster-data/ts-2/wal
--ranger_kms_url=invalid.host:1234/kms,127.19.52.84:44245/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517687384214-19665-0/krb5kdc/kudu_127.19.52.67.keytab
--principal=kudu/127.19.52.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.52.67:37273
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517687384214-19665-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.19.52.67
--webserver_port=42967
--tserver_master_addrs=127.19.52.126:44499
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2215
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.52.67
--log_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517687384214-19665-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 06:15:54 UTC on ccycloud.quasar-rnfzce.root.comops.site
build id 4581989
I20260812 06:57:39.033450 2215 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:57:39.034646 2215 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 06:57:39.046309 2225 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:57:39.046360 2222 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:57:39.046695 2223 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:57:39.046702 2215 server_base.cc:1034] running on GCE node
I20260812 06:57:39.047194 2215 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:57:39.048030 2215 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:57:39.049249 2215 hybrid_clock.cc:648] HybridClock initialized: now 1786517859049220 us; error 46 us; skew 500 ppm
Aug 12 06:57:39 dist-test-slave-65mx krb5kdc[707](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517859, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.52.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 06:57:39.053184 2215 init.cc:377] Logged in from keytab as kudu/127.19.52.67@KRBTEST.COM (short username kudu)
I20260812 06:57:39.054488 2215 webserver.cc:460] Webserver started at http://127.19.52.67:42967/ using document root <none> and password file <none>
I20260812 06:57:39.055240 2215 fs_manager.cc:359] Metadata directory not provided
I20260812 06:57:39.055351 2215 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 12 06:57:39 dist-test-slave-65mx krb5kdc[707](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517859, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.52.67@KRBTEST.COM for HTTP/127.19.52.84@KRBTEST.COM
I20260812 06:57:39.092207 2215 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.000s
I20260812 06:57:39.095327 2234 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 06:57:39.097205 2215 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.000s sys 0.002s
I20260812 06:57:39.097355 2215 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517687384214-19665-0/minicluster-data/ts-2/data,/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517687384214-19665-0/minicluster-data/ts-2/wal
uuid: "e203f3ed23644e688a664be089c903f1"
format_stamp: "Formatted at 2026-08-12 06:57:37 on dist-test-slave-65mx"
server_key: "8736fedb5cfcf88ce95913e6560b9361"
server_key_iv: "aff3a8b64bf3821850530bc6acd72aff"
server_key_version: "kuduclusterkey@0"
I20260812 06:57:39.097987 2215 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517687384214-19665-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517687384214-19665-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517687384214-19665-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:57:39.110476 2215 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 06:57:39.114313 2215 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 06:57:39.114553 2215 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:57:39.115293 2215 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 06:57:39.116313 2215 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 06:57:39.116391 2215 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 06:57:39.116463 2215 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 06:57:39.116510 2215 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 06:57:39.128259 2215 rpc_server.cc:307] RPC server started. Bound to: 127.19.52.67:37273
I20260812 06:57:39.128285 2372 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.52.67:37273 every 8 connection(s)
I20260812 06:57:39.129455 2215 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517687384214-19665-0/minicluster-data/ts-2/data/info.pb
I20260812 06:57:39.136072 19665 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu as pid 2215
Aug 12 06:57:39 dist-test-slave-65mx krb5kdc[707](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517859, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.52.67@KRBTEST.COM for kudu/127.19.52.126@KRBTEST.COM
I20260812 06:57:39.147512 2375 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:57:39.131980 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.126:44499 (local address 127.19.52.67:36187)
0812 06:57:39.132518 (+ 538us) negotiation.cc:107] Waiting for socket to connect
0812 06:57:39.132624 (+ 106us) client_negotiation.cc:174] Beginning negotiation
0812 06:57:39.133524 (+ 900us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:57:39.134159 (+ 635us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:57:39.134166 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:57:39.134453 (+ 287us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:57:39.134934 (+ 481us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:57:39.134947 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:57:39.136622 (+ 1675us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:57:39.136630 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:57:39.138131 (+ 1501us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:57:39.138144 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:57:39.138877 (+ 733us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:57:39.138882 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:57:39.139098 (+ 216us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:57:39.139869 (+ 771us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 06:57:39.139892 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:57:39.141512 (+ 1620us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 06:57:39.143777 (+ 2265us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:57:39.143783 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:57:39.143803 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:57:39.144239 (+ 436us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:57:39.144648 (+ 409us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:57:39.144651 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:57:39.144653 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:57:39.144868 (+ 215us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:57:39.145395 (+ 527us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 06:57:39.145403 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 06:57:39.145804 (+ 401us) client_negotiation.cc:769] Sending connection context
0812 06:57:39.146041 (+ 237us) client_negotiation.cc:240] Negotiation successful
0812 06:57:39.146367 (+ 326us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":339,"thread_start_us":156,"threads_started":1}
I20260812 06:57:39.148458 2061 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:57:39.131485 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.52.67:36187 (local address 127.19.52.126:44499)
0812 06:57:39.131626 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0812 06:57:39.131631 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 06:57:39.132744 (+ 1113us) server_negotiation.cc:407] Connection header received
0812 06:57:39.133759 (+ 1015us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:57:39.133763 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:57:39.133828 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:57:39.133933 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:57:39.135082 (+ 1149us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:57:39.135639 (+ 557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:57:39.138284 (+ 2645us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:57:39.138690 (+ 406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:57:39.138820 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:57:39.141702 (+ 2882us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:57:39.141706 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:57:39.141709 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:57:39.141742 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:57:39.143621 (+ 1879us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:57:39.144371 (+ 750us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:57:39.144375 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:57:39.144377 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:57:39.144482 (+ 105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:57:39.144996 (+ 514us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:57:39.144999 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:57:39.145001 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:57:39.145240 (+ 239us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:57:39.145335 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0812 06:57:39.148123 (+ 2788us) server_negotiation.cc:299] Negotiation successful
0812 06:57:39.148295 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260812 06:57:39.149811 2373 heartbeater.cc:347] Connected to a master server at 127.19.52.126:44499
I20260812 06:57:39.150120 2373 heartbeater.cc:464] Registering TS with master...
I20260812 06:57:39.150843 2373 heartbeater.cc:511] Master 127.19.52.126:44499 requested a full tablet report, sending...
W20260812 06:57:39.152742 2373 heartbeater.cc:644] Failed to heartbeat to 127.19.52.126:44499 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260812 06:57:39.152889 2373 heartbeater.cc:464] Registering TS with master...
I20260812 06:57:39.153096 2373 heartbeater.cc:511] Master 127.19.52.126:44499 requested a full tablet report, sending...
I20260812 06:57:39.154034 2373 heartbeater.cc:464] Registering TS with master...
I20260812 06:57:39.154222 2373 heartbeater.cc:511] Master 127.19.52.126:44499 requested a full tablet report, sending...
W20260812 06:57:39.155054 2373 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260812 06:57:39.160447 2377 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:57:39.138485 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39440 (local address 127.19.52.126:44499)
0812 06:57:39.138760 (+ 275us) server_negotiation.cc:206] Beginning negotiation
0812 06:57:39.138770 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0812 06:57:39.138784 (+ 14us) server_negotiation.cc:407] Connection header received
0812 06:57:39.138824 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:57:39.138827 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:57:39.138875 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:57:39.138978 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:57:39.140659 (+ 1681us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:57:39.141714 (+ 1055us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:57:39.148918 (+ 7204us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:57:39.149436 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:57:39.150032 (+ 596us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:57:39.156673 (+ 6641us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:57:39.156677 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:57:39.156680 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:57:39.156708 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:57:39.158446 (+ 1738us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:57:39.158982 (+ 536us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:57:39.158986 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:57:39.158988 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:57:39.159097 (+ 109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:57:39.159435 (+ 338us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:57:39.159438 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:57:39.159440 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:57:39.159679 (+ 239us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:57:39.159902 (+ 223us) server_negotiation.cc:1035] Waiting for connection context
0812 06:57:39.160115 (+ 213us) server_negotiation.cc:299] Negotiation successful
0812 06:57:39.160256 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":203,"thread_start_us":90,"threads_started":1}
Aug 12 06:57:39 dist-test-slave-65mx krb5kdc[707](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786517859, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.52.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 06:57:39.608275 2057 heartbeater.cc:464] Registering TS with master...
I20260812 06:57:39.608659 2057 heartbeater.cc:511] Master 127.19.52.126:44499 requested a full tablet report, sending...
I20260812 06:57:39.610111 1817 ts_manager.cc:194] Registered new tserver with Master: 97d4c4f7f749461f9119802f1c3a76f2 (127.19.52.65:37993)
I20260812 06:57:39.611778 1817 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.52.65@KRBTEST.COM'} at 127.19.52.65:34625
I20260812 06:57:39.880129 2212 heartbeater.cc:464] Registering TS with master...
I20260812 06:57:39.880483 2212 heartbeater.cc:511] Master 127.19.52.126:44499 requested a full tablet report, sending...
I20260812 06:57:39.881619 1817 ts_manager.cc:194] Registered new tserver with Master: bef89111ca8e428ea48be43c25f58d46 (127.19.52.66:38375)
I20260812 06:57:39.882401 1817 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.52.66@KRBTEST.COM'} at 127.19.52.66:42359
Aug 12 06:57:40 dist-test-slave-65mx krb5kdc[707](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517859, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.52.126@KRBTEST.COM for HTTP/127.19.52.84@KRBTEST.COM
I20260812 06:57:40.155364 2373 heartbeater.cc:464] Registering TS with master...
I20260812 06:57:40.155709 2373 heartbeater.cc:511] Master 127.19.52.126:44499 requested a full tablet report, sending...
I20260812 06:57:40.156654 1817 ts_manager.cc:194] Registered new tserver with Master: e203f3ed23644e688a664be089c903f1 (127.19.52.67:37273)
I20260812 06:57:40.157483 1817 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.52.67@KRBTEST.COM'} at 127.19.52.67:36187
I20260812 06:57:40.157682 19665 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 06:57:40.168658 2396 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:57:40.160267 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39446 (local address 127.19.52.126:44499)
0812 06:57:40.160522 (+ 255us) server_negotiation.cc:206] Beginning negotiation
0812 06:57:40.160527 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 06:57:40.160942 (+ 415us) server_negotiation.cc:407] Connection header received
0812 06:57:40.161113 (+ 171us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:57:40.161116 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:57:40.161165 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:57:40.161266 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:57:40.161959 (+ 693us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:57:40.162511 (+ 552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:57:40.163388 (+ 877us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:57:40.163640 (+ 252us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:57:40.163752 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:57:40.165241 (+ 1489us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:57:40.165245 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:57:40.165247 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:57:40.165273 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:57:40.166869 (+ 1596us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:57:40.167325 (+ 456us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:57:40.167329 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:57:40.167331 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:57:40.167416 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:57:40.167723 (+ 307us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:57:40.167726 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:57:40.167728 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:57:40.167913 (+ 185us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:57:40.168059 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0812 06:57:40.168310 (+ 251us) server_negotiation.cc:299] Negotiation successful
0812 06:57:40.168451 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":179,"thread_start_us":95,"threads_started":1}
I20260812 06:57:40.172753 1817 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39446:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 06:57:40.176232 1817 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 06:57:40.197773 2404 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:57:40.187219 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.67:37273 (local address 127.0.0.1:57852)
0812 06:57:40.189321 (+ 2102us) negotiation.cc:107] Waiting for socket to connect
0812 06:57:40.189378 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0812 06:57:40.189510 (+ 132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:57:40.192622 (+ 3112us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:57:40.192630 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:57:40.192656 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 06:57:40.192808 (+ 152us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:57:40.192815 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:57:40.194443 (+ 1628us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:57:40.194448 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:57:40.195807 (+ 1359us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:57:40.195817 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:57:40.197192 (+ 1375us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:57:40.197196 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:57:40.197304 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:57:40.197345 (+ 41us) client_negotiation.cc:769] Sending connection context
0812 06:57:40.197461 (+ 116us) client_negotiation.cc:240] Negotiation successful
0812 06:57:40.197600 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1048,"thread_start_us":157,"threads_started":1}
I20260812 06:57:40.198467 2406 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:57:40.188673 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57852 (local address 127.19.52.67:37273)
0812 06:57:40.191865 (+ 3192us) server_negotiation.cc:206] Beginning negotiation
0812 06:57:40.191874 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 06:57:40.191900 (+ 26us) server_negotiation.cc:407] Connection header received
0812 06:57:40.191998 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:57:40.192006 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:57:40.192251 (+ 245us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:57:40.192405 (+ 154us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 06:57:40.193018 (+ 613us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:57:40.194298 (+ 1280us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:57:40.195968 (+ 1670us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:57:40.196839 (+ 871us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:57:40.196966 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:57:40.197129 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0812 06:57:40.198207 (+ 1078us) server_negotiation.cc:299] Negotiation successful
0812 06:57:40.198319 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3081,"thread_start_us":104,"threads_started":1}
I20260812 06:57:40.200742 2403 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:57:40.187271 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.66:38375 (local address 127.0.0.1:48714)
0812 06:57:40.191480 (+ 4209us) negotiation.cc:107] Waiting for socket to connect
0812 06:57:40.191494 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 06:57:40.191590 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:57:40.192801 (+ 1211us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:57:40.192804 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:57:40.192819 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 06:57:40.192913 (+ 94us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:57:40.192920 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:57:40.195403 (+ 2483us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:57:40.195406 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:57:40.196687 (+ 1281us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:57:40.196698 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:57:40.200314 (+ 3616us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:57:40.200318 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:57:40.200407 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:57:40.200430 (+ 23us) client_negotiation.cc:769] Sending connection context
0812 06:57:40.200513 (+ 83us) client_negotiation.cc:240] Negotiation successful
0812 06:57:40.200609 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4135,"thread_start_us":105,"threads_started":1}
I20260812 06:57:40.201471 2407 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:57:40.189055 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48714 (local address 127.19.52.66:38375)
0812 06:57:40.192195 (+ 3140us) server_negotiation.cc:206] Beginning negotiation
0812 06:57:40.192201 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 06:57:40.192225 (+ 24us) server_negotiation.cc:407] Connection header received
0812 06:57:40.192294 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:57:40.192303 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:57:40.192477 (+ 174us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:57:40.192673 (+ 196us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 06:57:40.193310 (+ 637us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:57:40.194392 (+ 1082us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:57:40.198263 (+ 3871us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:57:40.199999 (+ 1736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:57:40.200107 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:57:40.200247 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0812 06:57:40.201215 (+ 968us) server_negotiation.cc:299] Negotiation successful
0812 06:57:40.201320 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3041,"thread_start_us":127,"threads_started":1}
I20260812 06:57:40.202061 2292 tablet_service.cc:1467] Processing CreateTablet for tablet 9626548d53cf42f5b6f146f7b5643ed4 (DEFAULT_TABLE table=test-table [id=a1eab79aaf0540f7a817bcdeb20f5d8b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 06:57:40.203301 2292 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9626548d53cf42f5b6f146f7b5643ed4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 06:57:40.204523 2134 tablet_service.cc:1467] Processing CreateTablet for tablet 9626548d53cf42f5b6f146f7b5643ed4 (DEFAULT_TABLE table=test-table [id=a1eab79aaf0540f7a817bcdeb20f5d8b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 06:57:40.205590 2134 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9626548d53cf42f5b6f146f7b5643ed4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 06:57:40.215102 2409 tablet_bootstrap.cc:492] T 9626548d53cf42f5b6f146f7b5643ed4 P e203f3ed23644e688a664be089c903f1: Bootstrap starting.
I20260812 06:57:40.217605 2409 tablet_bootstrap.cc:654] T 9626548d53cf42f5b6f146f7b5643ed4 P e203f3ed23644e688a664be089c903f1: Neither blocks nor log segments found. Creating new log.
I20260812 06:57:40.218909 2410 tablet_bootstrap.cc:492] T 9626548d53cf42f5b6f146f7b5643ed4 P bef89111ca8e428ea48be43c25f58d46: Bootstrap starting.
I20260812 06:57:40.219187 2408 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:57:40.189432 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42966 (local address 127.19.52.65:37993)
0812 06:57:40.197015 (+ 7583us) server_negotiation.cc:206] Beginning negotiation
0812 06:57:40.197023 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 06:57:40.197049 (+ 26us) server_negotiation.cc:407] Connection header received
0812 06:57:40.197131 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:57:40.197136 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:57:40.197368 (+ 232us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:57:40.197532 (+ 164us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 06:57:40.213287 (+ 15755us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:57:40.214488 (+ 1201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:57:40.217091 (+ 2603us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:57:40.218078 (+ 987us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:57:40.218166 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:57:40.218307 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0812 06:57:40.218923 (+ 616us) server_negotiation.cc:299] Negotiation successful
0812 06:57:40.219037 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":7428,"thread_start_us":97,"threads_started":1}
I20260812 06:57:40.219939 2402 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:57:40.188050 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.65:37993 (local address 127.0.0.1:42966)
0812 06:57:40.193113 (+ 5063us) negotiation.cc:107] Waiting for socket to connect
0812 06:57:40.193127 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 06:57:40.193204 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:57:40.197714 (+ 4510us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:57:40.197717 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:57:40.197734 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 06:57:40.197829 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:57:40.197837 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:57:40.214623 (+ 16786us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:57:40.214627 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:57:40.216882 (+ 2255us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:57:40.216897 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:57:40.218647 (+ 1750us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:57:40.218650 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:57:40.218746 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:57:40.218773 (+ 27us) client_negotiation.cc:769] Sending connection context
0812 06:57:40.219641 (+ 868us) client_negotiation.cc:240] Negotiation successful
0812 06:57:40.219776 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4999,"thread_start_us":114,"threads_started":1}
I20260812 06:57:40.221295 2410 tablet_bootstrap.cc:654] T 9626548d53cf42f5b6f146f7b5643ed4 P bef89111ca8e428ea48be43c25f58d46: Neither blocks nor log segments found. Creating new log.
I20260812 06:57:40.222122 2410 log.cc:824] T 9626548d53cf42f5b6f146f7b5643ed4 P bef89111ca8e428ea48be43c25f58d46: Log is configured to *not* fsync() on all Append() calls
I20260812 06:57:40.222666 2409 log.cc:824] T 9626548d53cf42f5b6f146f7b5643ed4 P e203f3ed23644e688a664be089c903f1: Log is configured to *not* fsync() on all Append() calls
I20260812 06:57:40.223327 1972 tablet_service.cc:1467] Processing CreateTablet for tablet 9626548d53cf42f5b6f146f7b5643ed4 (DEFAULT_TABLE table=test-table [id=a1eab79aaf0540f7a817bcdeb20f5d8b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 06:57:40.224479 1972 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9626548d53cf42f5b6f146f7b5643ed4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 06:57:40.225462 2409 tablet_bootstrap.cc:492] T 9626548d53cf42f5b6f146f7b5643ed4 P e203f3ed23644e688a664be089c903f1: No bootstrap required, opened a new log
I20260812 06:57:40.225663 2409 ts_tablet_manager.cc:1397] T 9626548d53cf42f5b6f146f7b5643ed4 P e203f3ed23644e688a664be089c903f1: Time spent bootstrapping tablet: real 0.011s user 0.005s sys 0.000s
I20260812 06:57:40.230201 2409 raft_consensus.cc:348] T 9626548d53cf42f5b6f146f7b5643ed4 P e203f3ed23644e688a664be089c903f1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bef89111ca8e428ea48be43c25f58d46" member_type: VOTER last_known_addr { host: "127.19.52.66" port: 38375 } } peers { permanent_uuid: "e203f3ed23644e688a664be089c903f1" member_type: VOTER last_known_addr { host: "127.19.52.67" port: 37273 } } peers { permanent_uuid: "97d4c4f7f749461f9119802f1c3a76f2" member_type: VOTER last_known_addr { host: "127.19.52.65" port: 37993 } }
I20260812 06:57:40.230660 2409 raft_consensus.cc:374] T 9626548d53cf42f5b6f146f7b5643ed4 P e203f3ed23644e688a664be089c903f1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 06:57:40.230732 2409 raft_consensus.cc:729] T 9626548d53cf42f5b6f146f7b5643ed4 P e203f3ed23644e688a664be089c903f1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e203f3ed23644e688a664be089c903f1, State: Initialized, Role: FOLLOWER
I20260812 06:57:40.231261 2409 consensus_queue.cc:260] T 9626548d53cf42f5b6f146f7b5643ed4 P e203f3ed23644e688a664be089c903f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bef89111ca8e428ea48be43c25f58d46" member_type: VOTER last_known_addr { host: "127.19.52.66" port: 38375 } } peers { permanent_uuid: "e203f3ed23644e688a664be089c903f1" member_type: VOTER last_known_addr { host: "127.19.52.67" port: 37273 } } peers { permanent_uuid: "97d4c4f7f749461f9119802f1c3a76f2" member_type: VOTER last_known_addr { host: "127.19.52.65" port: 37993 } }
I20260812 06:57:40.232636 2409 ts_tablet_manager.cc:1428] T 9626548d53cf42f5b6f146f7b5643ed4 P e203f3ed23644e688a664be089c903f1: Time spent starting tablet: real 0.007s user 0.008s sys 0.000s
I20260812 06:57:40.233057 2373 heartbeater.cc:503] Master 127.19.52.126:44499 was elected leader, sending a full tablet report...
I20260812 06:57:40.239816 2415 tablet_bootstrap.cc:492] T 9626548d53cf42f5b6f146f7b5643ed4 P 97d4c4f7f749461f9119802f1c3a76f2: Bootstrap starting.
I20260812 06:57:40.241103 2410 tablet_bootstrap.cc:492] T 9626548d53cf42f5b6f146f7b5643ed4 P bef89111ca8e428ea48be43c25f58d46: No bootstrap required, opened a new log
I20260812 06:57:40.241344 2410 ts_tablet_manager.cc:1397] T 9626548d53cf42f5b6f146f7b5643ed4 P bef89111ca8e428ea48be43c25f58d46: Time spent bootstrapping tablet: real 0.023s user 0.006s sys 0.000s
I20260812 06:57:40.242583 2415 tablet_bootstrap.cc:654] T 9626548d53cf42f5b6f146f7b5643ed4 P 97d4c4f7f749461f9119802f1c3a76f2: Neither blocks nor log segments found. Creating new log.
I20260812 06:57:40.243484 2415 log.cc:824] T 9626548d53cf42f5b6f146f7b5643ed4 P 97d4c4f7f749461f9119802f1c3a76f2: Log is configured to *not* fsync() on all Append() calls
I20260812 06:57:40.246284 2415 tablet_bootstrap.cc:492] T 9626548d53cf42f5b6f146f7b5643ed4 P 97d4c4f7f749461f9119802f1c3a76f2: No bootstrap required, opened a new log
I20260812 06:57:40.246482 2415 ts_tablet_manager.cc:1397] T 9626548d53cf42f5b6f146f7b5643ed4 P 97d4c4f7f749461f9119802f1c3a76f2: Time spent bootstrapping tablet: real 0.007s user 0.004s sys 0.000s
I20260812 06:57:40.246452 2410 raft_consensus.cc:348] T 9626548d53cf42f5b6f146f7b5643ed4 P bef89111ca8e428ea48be43c25f58d46 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bef89111ca8e428ea48be43c25f58d46" member_type: VOTER last_known_addr { host: "127.19.52.66" port: 38375 } } peers { permanent_uuid: "e203f3ed23644e688a664be089c903f1" member_type: VOTER last_known_addr { host: "127.19.52.67" port: 37273 } } peers { permanent_uuid: "97d4c4f7f749461f9119802f1c3a76f2" member_type: VOTER last_known_addr { host: "127.19.52.65" port: 37993 } }
I20260812 06:57:40.246768 2410 raft_consensus.cc:374] T 9626548d53cf42f5b6f146f7b5643ed4 P bef89111ca8e428ea48be43c25f58d46 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 06:57:40.246830 2410 raft_consensus.cc:729] T 9626548d53cf42f5b6f146f7b5643ed4 P bef89111ca8e428ea48be43c25f58d46 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bef89111ca8e428ea48be43c25f58d46, State: Initialized, Role: FOLLOWER
I20260812 06:57:40.247364 2410 consensus_queue.cc:260] T 9626548d53cf42f5b6f146f7b5643ed4 P bef89111ca8e428ea48be43c25f58d46 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bef89111ca8e428ea48be43c25f58d46" member_type: VOTER last_known_addr { host: "127.19.52.66" port: 38375 } } peers { permanent_uuid: "e203f3ed23644e688a664be089c903f1" member_type: VOTER last_known_addr { host: "127.19.52.67" port: 37273 } } peers { permanent_uuid: "97d4c4f7f749461f9119802f1c3a76f2" member_type: VOTER last_known_addr { host: "127.19.52.65" port: 37993 } }
I20260812 06:57:40.248538 2410 ts_tablet_manager.cc:1428] T 9626548d53cf42f5b6f146f7b5643ed4 P bef89111ca8e428ea48be43c25f58d46: Time spent starting tablet: real 0.007s user 0.007s sys 0.000s
I20260812 06:57:40.249111 2212 heartbeater.cc:503] Master 127.19.52.126:44499 was elected leader, sending a full tablet report...
I20260812 06:57:40.250723 2415 raft_consensus.cc:348] T 9626548d53cf42f5b6f146f7b5643ed4 P 97d4c4f7f749461f9119802f1c3a76f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bef89111ca8e428ea48be43c25f58d46" member_type: VOTER last_known_addr { host: "127.19.52.66" port: 38375 } } peers { permanent_uuid: "e203f3ed23644e688a664be089c903f1" member_type: VOTER last_known_addr { host: "127.19.52.67" port: 37273 } } peers { permanent_uuid: "97d4c4f7f749461f9119802f1c3a76f2" member_type: VOTER last_known_addr { host: "127.19.52.65" port: 37993 } }
I20260812 06:57:40.251051 2415 raft_consensus.cc:374] T 9626548d53cf42f5b6f146f7b5643ed4 P 97d4c4f7f749461f9119802f1c3a76f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 06:57:40.251121 2415 raft_consensus.cc:729] T 9626548d53cf42f5b6f146f7b5643ed4 P 97d4c4f7f749461f9119802f1c3a76f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 97d4c4f7f749461f9119802f1c3a76f2, State: Initialized, Role: FOLLOWER
I20260812 06:57:40.251703 2415 consensus_queue.cc:260] T 9626548d53cf42f5b6f146f7b5643ed4 P 97d4c4f7f749461f9119802f1c3a76f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bef89111ca8e428ea48be43c25f58d46" member_type: VOTER last_known_addr { host: "127.19.52.66" port: 38375 } } peers { permanent_uuid: "e203f3ed23644e688a664be089c903f1" member_type: VOTER last_known_addr { host: "127.19.52.67" port: 37273 } } peers { permanent_uuid: "97d4c4f7f749461f9119802f1c3a76f2" member_type: VOTER last_known_addr { host: "127.19.52.65" port: 37993 } }
I20260812 06:57:40.253070 2415 ts_tablet_manager.cc:1428] T 9626548d53cf42f5b6f146f7b5643ed4 P 97d4c4f7f749461f9119802f1c3a76f2: Time spent starting tablet: real 0.007s user 0.008s sys 0.000s
I20260812 06:57:40.253476 2057 heartbeater.cc:503] Master 127.19.52.126:44499 was elected leader, sending a full tablet report...
W20260812 06:57:40.314585 2058 tablet.cc:2369] T 9626548d53cf42f5b6f146f7b5643ed4 P 97d4c4f7f749461f9119802f1c3a76f2: Can't schedule compaction. Clean time has not been advanced past its initial value.
Aug 12 06:57:40 dist-test-slave-65mx krb5kdc[707](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517859, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.52.126@KRBTEST.COM for HTTP/127.19.52.84@KRBTEST.COM
W20260812 06:57:40.344857 2213 tablet.cc:2369] T 9626548d53cf42f5b6f146f7b5643ed4 P bef89111ca8e428ea48be43c25f58d46: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 06:57:40.381563 2374 tablet.cc:2369] T 9626548d53cf42f5b6f146f7b5643ed4 P e203f3ed23644e688a664be089c903f1: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 06:57:40.467295 2420 raft_consensus.cc:482] T 9626548d53cf42f5b6f146f7b5643ed4 P 97d4c4f7f749461f9119802f1c3a76f2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 06:57:40.467566 2420 raft_consensus.cc:504] T 9626548d53cf42f5b6f146f7b5643ed4 P 97d4c4f7f749461f9119802f1c3a76f2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bef89111ca8e428ea48be43c25f58d46" member_type: VOTER last_known_addr { host: "127.19.52.66" port: 38375 } } peers { permanent_uuid: "e203f3ed23644e688a664be089c903f1" member_type: VOTER last_known_addr { host: "127.19.52.67" port: 37273 } } peers { permanent_uuid: "97d4c4f7f749461f9119802f1c3a76f2" member_type: VOTER last_known_addr { host: "127.19.52.65" port: 37993 } }
I20260812 06:57:40.469118 2420 leader_election.cc:288] T 9626548d53cf42f5b6f146f7b5643ed4 P 97d4c4f7f749461f9119802f1c3a76f2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bef89111ca8e428ea48be43c25f58d46 (127.19.52.66:38375), e203f3ed23644e688a664be089c903f1 (127.19.52.67:37273)
I20260812 06:57:40.479264 2407 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:57:40.470532 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.52.65:59181 (local address 127.19.52.66:38375)
0812 06:57:40.470855 (+ 323us) server_negotiation.cc:206] Beginning negotiation
0812 06:57:40.470861 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 06:57:40.471120 (+ 259us) server_negotiation.cc:407] Connection header received
0812 06:57:40.471346 (+ 226us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:57:40.471350 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:57:40.471409 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:57:40.471467 (+ 58us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 06:57:40.471997 (+ 530us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:57:40.472820 (+ 823us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:57:40.477796 (+ 4976us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:57:40.478426 (+ 630us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:57:40.478542 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:57:40.478614 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0812 06:57:40.479007 (+ 393us) server_negotiation.cc:299] Negotiation successful
0812 06:57:40.479108 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":231}
I20260812 06:57:40.479820 2424 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:57:40.469548 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.66:38375 (local address 127.19.52.65:59181)
0812 06:57:40.471022 (+ 1474us) negotiation.cc:107] Waiting for socket to connect
0812 06:57:40.471037 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 06:57:40.471209 (+ 172us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:57:40.471640 (+ 431us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:57:40.471644 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:57:40.471664 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 06:57:40.471816 (+ 152us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:57:40.471824 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:57:40.472978 (+ 1154us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:57:40.472982 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:57:40.474327 (+ 1345us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:57:40.474340 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:57:40.478723 (+ 4383us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:57:40.478726 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:57:40.478835 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:57:40.478861 (+ 26us) client_negotiation.cc:769] Sending connection context
0812 06:57:40.479548 (+ 687us) client_negotiation.cc:240] Negotiation successful
0812 06:57:40.479652 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1403,"thread_start_us":124,"threads_started":1}
I20260812 06:57:40.480608 2158 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9626548d53cf42f5b6f146f7b5643ed4" candidate_uuid: "97d4c4f7f749461f9119802f1c3a76f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bef89111ca8e428ea48be43c25f58d46" is_pre_election: true
I20260812 06:57:40.480978 2158 raft_consensus.cc:2454] T 9626548d53cf42f5b6f146f7b5643ed4 P bef89111ca8e428ea48be43c25f58d46 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 97d4c4f7f749461f9119802f1c3a76f2 in term 0.
I20260812 06:57:40.481603 1917 leader_election.cc:302] T 9626548d53cf42f5b6f146f7b5643ed4 P 97d4c4f7f749461f9119802f1c3a76f2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 97d4c4f7f749461f9119802f1c3a76f2, bef89111ca8e428ea48be43c25f58d46; no voters:
I20260812 06:57:40.481927 2406 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:57:40.470703 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.52.65:37921 (local address 127.19.52.67:37273)
0812 06:57:40.470859 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0812 06:57:40.470864 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 06:57:40.474782 (+ 3918us) server_negotiation.cc:407] Connection header received
0812 06:57:40.474997 (+ 215us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:57:40.475001 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:57:40.475052 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:57:40.475103 (+ 51us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 06:57:40.475504 (+ 401us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:57:40.476268 (+ 764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:57:40.480518 (+ 4250us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:57:40.481163 (+ 645us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:57:40.481267 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:57:40.481318 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0812 06:57:40.481706 (+ 388us) server_negotiation.cc:299] Negotiation successful
0812 06:57:40.481805 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260812 06:57:40.481977 2420 raft_consensus.cc:2781] T 9626548d53cf42f5b6f146f7b5643ed4 P 97d4c4f7f749461f9119802f1c3a76f2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 06:57:40.482084 2420 raft_consensus.cc:482] T 9626548d53cf42f5b6f146f7b5643ed4 P 97d4c4f7f749461f9119802f1c3a76f2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 06:57:40.482127 2420 raft_consensus.cc:3037] T 9626548d53cf42f5b6f146f7b5643ed4 P 97d4c4f7f749461f9119802f1c3a76f2 [term 0 FOLLOWER]: Advancing to term 1
I20260812 06:57:40.482378 2423 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:57:40.470014 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.67:37273 (local address 127.19.52.65:37921)
0812 06:57:40.474676 (+ 4662us) negotiation.cc:107] Waiting for socket to connect
0812 06:57:40.474692 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0812 06:57:40.474865 (+ 173us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:57:40.475239 (+ 374us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:57:40.475242 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:57:40.475259 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 06:57:40.475369 (+ 110us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:57:40.475375 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:57:40.476424 (+ 1049us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:57:40.476427 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:57:40.480295 (+ 3868us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:57:40.480307 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:57:40.481423 (+ 1116us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:57:40.481427 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:57:40.481532 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:57:40.481555 (+ 23us) client_negotiation.cc:769] Sending connection context
0812 06:57:40.482138 (+ 583us) client_negotiation.cc:240] Negotiation successful
0812 06:57:40.482230 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4582,"thread_start_us":100,"threads_started":1}
I20260812 06:57:40.483023 2317 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9626548d53cf42f5b6f146f7b5643ed4" candidate_uuid: "97d4c4f7f749461f9119802f1c3a76f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e203f3ed23644e688a664be089c903f1" is_pre_election: true
I20260812 06:57:40.483356 2317 raft_consensus.cc:2454] T 9626548d53cf42f5b6f146f7b5643ed4 P e203f3ed23644e688a664be089c903f1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 97d4c4f7f749461f9119802f1c3a76f2 in term 0.
I20260812 06:57:40.483526 2420 raft_consensus.cc:504] T 9626548d53cf42f5b6f146f7b5643ed4 P 97d4c4f7f749461f9119802f1c3a76f2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bef89111ca8e428ea48be43c25f58d46" member_type: VOTER last_known_addr { host: "127.19.52.66" port: 38375 } } peers { permanent_uuid: "e203f3ed23644e688a664be089c903f1" member_type: VOTER last_known_addr { host: "127.19.52.67" port: 37273 } } peers { permanent_uuid: "97d4c4f7f749461f9119802f1c3a76f2" member_type: VOTER last_known_addr { host: "127.19.52.65" port: 37993 } }
I20260812 06:57:40.483906 2420 leader_election.cc:288] T 9626548d53cf42f5b6f146f7b5643ed4 P 97d4c4f7f749461f9119802f1c3a76f2 [CANDIDATE]: Term 1 election: Requested vote from peers bef89111ca8e428ea48be43c25f58d46 (127.19.52.66:38375), e203f3ed23644e688a664be089c903f1 (127.19.52.67:37273)
I20260812 06:57:40.484438 2317 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9626548d53cf42f5b6f146f7b5643ed4" candidate_uuid: "97d4c4f7f749461f9119802f1c3a76f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e203f3ed23644e688a664be089c903f1"
I20260812 06:57:40.484522 2158 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9626548d53cf42f5b6f146f7b5643ed4" candidate_uuid: "97d4c4f7f749461f9119802f1c3a76f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bef89111ca8e428ea48be43c25f58d46"
I20260812 06:57:40.484601 2317 raft_consensus.cc:3037] T 9626548d53cf42f5b6f146f7b5643ed4 P e203f3ed23644e688a664be089c903f1 [term 0 FOLLOWER]: Advancing to term 1
I20260812 06:57:40.484709 2158 raft_consensus.cc:3037] T 9626548d53cf42f5b6f146f7b5643ed4 P bef89111ca8e428ea48be43c25f58d46 [term 0 FOLLOWER]: Advancing to term 1
I20260812 06:57:40.485996 2158 raft_consensus.cc:2454] T 9626548d53cf42f5b6f146f7b5643ed4 P bef89111ca8e428ea48be43c25f58d46 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 97d4c4f7f749461f9119802f1c3a76f2 in term 1.
I20260812 06:57:40.485997 2317 raft_consensus.cc:2454] T 9626548d53cf42f5b6f146f7b5643ed4 P e203f3ed23644e688a664be089c903f1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 97d4c4f7f749461f9119802f1c3a76f2 in term 1.
I20260812 06:57:40.486379 1914 leader_election.cc:302] T 9626548d53cf42f5b6f146f7b5643ed4 P 97d4c4f7f749461f9119802f1c3a76f2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 97d4c4f7f749461f9119802f1c3a76f2, e203f3ed23644e688a664be089c903f1; no voters:
I20260812 06:57:40.486713 2420 raft_consensus.cc:2781] T 9626548d53cf42f5b6f146f7b5643ed4 P 97d4c4f7f749461f9119802f1c3a76f2 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 06:57:40.486976 2420 raft_consensus.cc:686] T 9626548d53cf42f5b6f146f7b5643ed4 P 97d4c4f7f749461f9119802f1c3a76f2 [term 1 LEADER]: Becoming Leader. State: Replica: 97d4c4f7f749461f9119802f1c3a76f2, State: Running, Role: LEADER
I20260812 06:57:40.487368 2420 consensus_queue.cc:237] T 9626548d53cf42f5b6f146f7b5643ed4 P 97d4c4f7f749461f9119802f1c3a76f2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bef89111ca8e428ea48be43c25f58d46" member_type: VOTER last_known_addr { host: "127.19.52.66" port: 38375 } } peers { permanent_uuid: "e203f3ed23644e688a664be089c903f1" member_type: VOTER last_known_addr { host: "127.19.52.67" port: 37273 } } peers { permanent_uuid: "97d4c4f7f749461f9119802f1c3a76f2" member_type: VOTER last_known_addr { host: "127.19.52.65" port: 37993 } }
I20260812 06:57:40.491104 1817 catalog_manager.cc:5495] T 9626548d53cf42f5b6f146f7b5643ed4 P 97d4c4f7f749461f9119802f1c3a76f2 reported cstate change: term changed from 0 to 1, leader changed from <none> to 97d4c4f7f749461f9119802f1c3a76f2 (127.19.52.65). New cstate: current_term: 1 leader_uuid: "97d4c4f7f749461f9119802f1c3a76f2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bef89111ca8e428ea48be43c25f58d46" member_type: VOTER last_known_addr { host: "127.19.52.66" port: 38375 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e203f3ed23644e688a664be089c903f1" member_type: VOTER last_known_addr { host: "127.19.52.67" port: 37273 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "97d4c4f7f749461f9119802f1c3a76f2" member_type: VOTER last_known_addr { host: "127.19.52.65" port: 37993 } health_report { overall_health: HEALTHY } } }
I20260812 06:57:40.508615 2408 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:57:40.503198 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42980 (local address 127.19.52.65:37993)
0812 06:57:40.503320 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0812 06:57:40.503324 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 06:57:40.503337 (+ 13us) server_negotiation.cc:407] Connection header received
0812 06:57:40.503431 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:57:40.503434 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:57:40.503505 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:57:40.503626 (+ 121us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 06:57:40.504072 (+ 446us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:57:40.504933 (+ 861us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:57:40.506235 (+ 1302us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:57:40.506599 (+ 364us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:57:40.506702 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:57:40.507168 (+ 466us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 06:57:40.507757 (+ 589us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 06:57:40.507884 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0812 06:57:40.508339 (+ 455us) server_negotiation.cc:299] Negotiation successful
0812 06:57:40.508407 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260812 06:57:40.516603 2317 raft_consensus.cc:1261] T 9626548d53cf42f5b6f146f7b5643ed4 P e203f3ed23644e688a664be089c903f1 [term 1 FOLLOWER]: Refusing update from remote peer 97d4c4f7f749461f9119802f1c3a76f2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 06:57:40.517769 2425 consensus_queue.cc:1035] T 9626548d53cf42f5b6f146f7b5643ed4 P 97d4c4f7f749461f9119802f1c3a76f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e203f3ed23644e688a664be089c903f1" member_type: VOTER last_known_addr { host: "127.19.52.67" port: 37273 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 06:57:40.525590 2158 raft_consensus.cc:1261] T 9626548d53cf42f5b6f146f7b5643ed4 P bef89111ca8e428ea48be43c25f58d46 [term 1 FOLLOWER]: Refusing update from remote peer 97d4c4f7f749461f9119802f1c3a76f2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 06:57:40.526341 2425 consensus_queue.cc:1035] T 9626548d53cf42f5b6f146f7b5643ed4 P 97d4c4f7f749461f9119802f1c3a76f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bef89111ca8e428ea48be43c25f58d46" member_type: VOTER last_known_addr { host: "127.19.52.66" port: 38375 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 06:57:40.540998 2428 mvcc.cc:205] Tried to move back new op lower bound from 7317577156665925632 to 7317577156559896576. Current Snapshot: MvccSnapshot[applied={T|T < 7317577156665925632}]
I20260812 06:57:40.541503 2429 mvcc.cc:205] Tried to move back new op lower bound from 7317577156665925632 to 7317577156559896576. Current Snapshot: MvccSnapshot[applied={T|T < 7317577156665925632}]
I20260812 06:57:40.553633 2431 mvcc.cc:205] Tried to move back new op lower bound from 7317577156665925632 to 7317577156559896576. Current Snapshot: MvccSnapshot[applied={T|T < 7317577156665925632}]
I20260812 06:57:40.557780 1817 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39446:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 06:57:40.558012 1817 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39446:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 06:57:40.561247 1817 catalog_manager.cc:5782] T 00000000000000000000000000000000 P cef00b9741644f17991e64877f67081d: Sending DeleteTablet for 3 replicas of tablet 9626548d53cf42f5b6f146f7b5643ed4
I20260812 06:57:40.569583 2134 tablet_service.cc:1514] Processing DeleteTablet for tablet 9626548d53cf42f5b6f146f7b5643ed4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 06:57:40 UTC) from {username='kudu', principal='kudu/127.19.52.126@KRBTEST.COM'} at 127.0.0.1:48714
I20260812 06:57:40.570174 1972 tablet_service.cc:1514] Processing DeleteTablet for tablet 9626548d53cf42f5b6f146f7b5643ed4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 06:57:40 UTC) from {username='kudu', principal='kudu/127.19.52.126@KRBTEST.COM'} at 127.0.0.1:42966
I20260812 06:57:40.570814 2292 tablet_service.cc:1514] Processing DeleteTablet for tablet 9626548d53cf42f5b6f146f7b5643ed4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 06:57:40 UTC) from {username='kudu', principal='kudu/127.19.52.126@KRBTEST.COM'} at 127.0.0.1:57852
I20260812 06:57:40.571949 19665 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu with pid 1860
I20260812 06:57:40.576858 2440 tablet_replica.cc:330] T 9626548d53cf42f5b6f146f7b5643ed4 P bef89111ca8e428ea48be43c25f58d46: stopping tablet replica
I20260812 06:57:40.577675 2440 raft_consensus.cc:2229] T 9626548d53cf42f5b6f146f7b5643ed4 P bef89111ca8e428ea48be43c25f58d46 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 06:57:40.578090 2440 raft_consensus.cc:2258] T 9626548d53cf42f5b6f146f7b5643ed4 P bef89111ca8e428ea48be43c25f58d46 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 06:57:40.580255 2440 ts_tablet_manager.cc:1905] T 9626548d53cf42f5b6f146f7b5643ed4 P bef89111ca8e428ea48be43c25f58d46: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 06:57:40.588231 2442 tablet_replica.cc:330] T 9626548d53cf42f5b6f146f7b5643ed4 P e203f3ed23644e688a664be089c903f1: stopping tablet replica
I20260812 06:57:40.588809 2442 raft_consensus.cc:2229] T 9626548d53cf42f5b6f146f7b5643ed4 P e203f3ed23644e688a664be089c903f1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 06:57:40.589196 2442 raft_consensus.cc:2258] T 9626548d53cf42f5b6f146f7b5643ed4 P e203f3ed23644e688a664be089c903f1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 06:57:40.589593 2440 ts_tablet_manager.cc:1918] T 9626548d53cf42f5b6f146f7b5643ed4 P bef89111ca8e428ea48be43c25f58d46: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 06:57:40.589728 2440 log.cc:1196] T 9626548d53cf42f5b6f146f7b5643ed4 P bef89111ca8e428ea48be43c25f58d46: Deleting WAL directory at /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517687384214-19665-0/minicluster-data/ts-1/wal/wals/9626548d53cf42f5b6f146f7b5643ed4
I20260812 06:57:40.590545 2440 ts_tablet_manager.cc:1939] T 9626548d53cf42f5b6f146f7b5643ed4 P bef89111ca8e428ea48be43c25f58d46: Deleting consensus metadata
I20260812 06:57:40.591251 2442 ts_tablet_manager.cc:1905] T 9626548d53cf42f5b6f146f7b5643ed4 P e203f3ed23644e688a664be089c903f1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 06:57:40.591977 1803 catalog_manager.cc:4841] TS bef89111ca8e428ea48be43c25f58d46 (127.19.52.66:38375): tablet 9626548d53cf42f5b6f146f7b5643ed4 (table test-table [id=a1eab79aaf0540f7a817bcdeb20f5d8b]) successfully deleted
I20260812 06:57:40.594422 2442 ts_tablet_manager.cc:1918] T 9626548d53cf42f5b6f146f7b5643ed4 P e203f3ed23644e688a664be089c903f1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 06:57:40.594543 2442 log.cc:1196] T 9626548d53cf42f5b6f146f7b5643ed4 P e203f3ed23644e688a664be089c903f1: Deleting WAL directory at /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517687384214-19665-0/minicluster-data/ts-2/wal/wals/9626548d53cf42f5b6f146f7b5643ed4
I20260812 06:57:40.594987 2442 ts_tablet_manager.cc:1939] T 9626548d53cf42f5b6f146f7b5643ed4 P e203f3ed23644e688a664be089c903f1: Deleting consensus metadata
W20260812 06:57:40.595757 1803 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.19.52.65:37993 (error 108)
I20260812 06:57:40.598177 1800 catalog_manager.cc:4841] TS e203f3ed23644e688a664be089c903f1 (127.19.52.67:37273): tablet 9626548d53cf42f5b6f146f7b5643ed4 (table test-table [id=a1eab79aaf0540f7a817bcdeb20f5d8b]) successfully deleted
I20260812 06:57:40.598577 19665 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu with pid 2059
I20260812 06:57:40.608875 2402 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:57:40.608361 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.65:37993 (local address 127.0.0.1:42990)
0812 06:57:40.608650 (+ 289us) negotiation.cc:107] Waiting for socket to connect
0812 06:57:40.608748 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.52.65:37993: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":145}
W20260812 06:57:40.609170 1803 catalog_manager.cc:4584] TS 97d4c4f7f749461f9119802f1c3a76f2 (127.19.52.65:37993): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 9626548d53cf42f5b6f146f7b5643ed4: Network error: Client connection negotiation failed: client connection to 127.19.52.65:37993: connect: Connection refused (error 111)
I20260812 06:57:40.612833 19665 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu with pid 2215
I20260812 06:57:40.628676 19665 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu with pid 1783
I20260812 06:57:40.643951 19665 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260812 06:57:40.990022 19665 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260812 06:57:40.990191 19665 mini_ranger.cc:66] Stopping Ranger...
I20260812 06:57:41.336349 19665 mini_ranger.cc:68] Stopped Ranger
2026-08-12 06:57:41.336 UTC [739] LOG: received smart shutdown request
2026-08-12 06:57:41.341 UTC [739] LOG: background worker "logical replication launcher" (PID 749) exited with exit code 1
2026-08-12 06:57:41.341 UTC [743] LOG: shutting down
2026-08-12 06:57:41.570 UTC [739] LOG: database system is shut down
2026-08-12T06:57:41Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers (53378 ms)
[ RUN ] SecurityITest.IPKICACert
Loading random data
Initializing database '/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786517687384214-19665-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 06:57:41 dist-test-slave-65mx krb5kdc[2453](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 06:57:41 dist-test-slave-65mx krb5kdc[2453](info): set up 2 sockets
Aug 12 06:57:41 dist-test-slave-65mx krb5kdc[2453](info): commencing operation
krb5kdc: starting...
W20260812 06:57:42.878480 19665 mini_kdc.cc:121] Time spent starting KDC: real 1.143s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786517687384214-19665-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786517687384214-19665-0/krb5kdc/test-user.keytab.
Aug 12 06:57:42 dist-test-slave-65mx krb5kdc[2453](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517862, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T06:57:42Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T06:57:42Z Disabled control of system clock
WARNING: no policy specified for kudu/127.19.52.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.52.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.52.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786517687384214-19665-0/krb5kdc/kudu_127.19.52.126.keytab.
Entry for principal kudu/127.19.52.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786517687384214-19665-0/krb5kdc/kudu_127.19.52.126.keytab.
I20260812 06:57:43.020766 19665 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu
/tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786517687384214-19665-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786517687384214-19665-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786517687384214-19665-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786517687384214-19665-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.52.126:46371
--webserver_interface=127.19.52.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786517687384214-19665-0/krb5kdc/kudu_127.19.52.126.keytab
--principal=kudu/127.19.52.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.52.84:33491
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.19.52.126:46371,127.19.52.125:34425,127.19.52.124:39693
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786517687384214-19665-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786517687384214-19665-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786517687384214-19665-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 06:57:43.138964 2470 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:57:43.139269 2470 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:57:43.139313 2470 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 06:57:43.143610 2470 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 06:57:43.143695 2470 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 06:57:43.143715 2470 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 06:57:43.143733 2470 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 06:57:43.143749 2470 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 06:57:43.149693 2470 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.52.84:33491
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786517687384214-19665-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786517687384214-19665-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.19.52.126:46371,127.19.52.125:34425,127.19.52.124:39693
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786517687384214-19665-0/krb5kdc/kudu_127.19.52.126.keytab
--principal=kudu/127.19.52.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.52.126:46371
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786517687384214-19665-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.19.52.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2470
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786517687384214-19665-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 06:15:54 UTC on ccycloud.quasar-rnfzce.root.comops.site
build id 4581989
I20260812 06:57:43.150931 2470 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:57:43.152045 2470 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 06:57:43.161226 2480 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:57:43.161218 2476 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:57:43.161244 2477 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:57:43.161676 2470 server_base.cc:1034] running on GCE node
I20260812 06:57:43.162384 2470 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:57:43.163659 2470 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:57:43.164870 2470 hybrid_clock.cc:648] HybridClock initialized: now 1786517863164839 us; error 42 us; skew 500 ppm
Aug 12 06:57:43 dist-test-slave-65mx krb5kdc[2453](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517863, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.52.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 06:57:43.167915 2470 init.cc:377] Logged in from keytab as kudu/127.19.52.126@KRBTEST.COM (short username kudu)
I20260812 06:57:43.169009 2470 webserver.cc:460] Webserver started at http://127.19.52.126:35541/ using document root <none> and password file <none>
I20260812 06:57:43.169616 2470 fs_manager.cc:359] Metadata directory not provided
I20260812 06:57:43.169714 2470 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:57:43.169963 2470 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 06:57:43.172102 2470 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786517687384214-19665-0/minicluster-data/master-0/data/instance:
uuid: "54ebea6e8de44f078f2e8b3885cf7237"
format_stamp: "Formatted at 2026-08-12 06:57:43 on dist-test-slave-65mx"
server_key: "045c0572a61a3261ee352f4433ce4440"
server_key_iv: "3ddb1469bdfa3ccdd72567e0afc56c2e"
server_key_version: "encryptionkey@0"
I20260812 06:57:43.172766 2470 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786517687384214-19665-0/minicluster-data/master-0/wal/instance:
uuid: "54ebea6e8de44f078f2e8b3885cf7237"
format_stamp: "Formatted at 2026-08-12 06:57:43 on dist-test-slave-65mx"
server_key: "045c0572a61a3261ee352f4433ce4440"
server_key_iv: "3ddb1469bdfa3ccdd72567e0afc56c2e"
server_key_version: "encryptionkey@0"
I20260812 06:57:43.176694 2470 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260812 06:57:43.179394 2488 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 06:57:43.180747 2470 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 06:57:43.180894 2470 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786517687384214-19665-0/minicluster-data/master-0/data,/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786517687384214-19665-0/minicluster-data/master-0/wal
uuid: "54ebea6e8de44f078f2e8b3885cf7237"
format_stamp: "Formatted at 2026-08-12 06:57:43 on dist-test-slave-65mx"
server_key: "045c0572a61a3261ee352f4433ce4440"
server_key_iv: "3ddb1469bdfa3ccdd72567e0afc56c2e"
server_key_version: "encryptionkey@0"
I20260812 06:57:43.181017 2470 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786517687384214-19665-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786517687384214-19665-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786517687384214-19665-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:57:43.205545 2470 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 06:57:43.208935 2470 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 06:57:43.209167 2470 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:57:43.216663 2470 rpc_server.cc:307] RPC server started. Bound to: 127.19.52.126:46371
I20260812 06:57:43.216683 2544 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.52.126:46371 every 8 connection(s)
I20260812 06:57:43.217696 2470 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786517687384214-19665-0/minicluster-data/master-0/data/info.pb
I20260812 06:57:43.219115 19665 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu as pid 2470
I20260812 06:57:43.219255 19665 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786517687384214-19665-0/minicluster-data/master-0/wal/instance
I20260812 06:57:43.219592 19665 external_mini_cluster.cc:1428] Setting key 2e762f588c30184bc41f056e19e46e6a
I20260812 06:57:43.221189 2545 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 06:57:43.225739 2545 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.19.52.126" port: 46371 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.19.52.125@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.52.125@KRBTEST.COM" created.
Aug 12 06:57:43 dist-test-slave-65mx krb5kdc[2453](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517863, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.52.126@KRBTEST.COM for kudu/127.19.52.126@KRBTEST.COM
I20260812 06:57:43.243167 2549 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:57:43.228326 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.126:46371 (local address 127.0.0.1:40506)
0812 06:57:43.229058 (+ 732us) negotiation.cc:107] Waiting for socket to connect
0812 06:57:43.229095 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0812 06:57:43.229978 (+ 883us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:57:43.230688 (+ 710us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:57:43.230693 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:57:43.231132 (+ 439us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:57:43.231415 (+ 283us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:57:43.231429 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:57:43.233140 (+ 1711us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:57:43.233148 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:57:43.233878 (+ 730us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:57:43.233886 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:57:43.234835 (+ 949us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:57:43.234842 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:57:43.235006 (+ 164us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:57:43.235860 (+ 854us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 06:57:43.235887 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:57:43.237560 (+ 1673us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 06:57:43.239822 (+ 2262us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:57:43.239833 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:57:43.239844 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:57:43.240113 (+ 269us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:57:43.240467 (+ 354us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:57:43.240470 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:57:43.240472 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:57:43.240655 (+ 183us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:57:43.241355 (+ 700us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 06:57:43.241362 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 06:57:43.241515 (+ 153us) client_negotiation.cc:769] Sending connection context
0812 06:57:43.241688 (+ 173us) client_negotiation.cc:240] Negotiation successful
0812 06:57:43.242101 (+ 413us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":560,"mutex_wait_us":116,"spinlock_wait_cycles":27392,"thread_start_us":206,"threads_started":1}
I20260812 06:57:43.243167 2548 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:57:43.228325 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40506 (local address 127.19.52.126:46371)
0812 06:57:43.228832 (+ 507us) server_negotiation.cc:206] Beginning negotiation
0812 06:57:43.228852 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0812 06:57:43.229205 (+ 353us) server_negotiation.cc:407] Connection header received
0812 06:57:43.230316 (+ 1111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:57:43.230321 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:57:43.230504 (+ 183us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:57:43.230691 (+ 187us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:57:43.231640 (+ 949us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:57:43.232997 (+ 1357us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:57:43.234009 (+ 1012us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:57:43.234630 (+ 621us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:57:43.234961 (+ 331us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:57:43.237730 (+ 2769us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:57:43.237737 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:57:43.237752 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:57:43.237782 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:57:43.239683 (+ 1901us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:57:43.240234 (+ 551us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:57:43.240240 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:57:43.240242 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:57:43.240355 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:57:43.240756 (+ 401us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:57:43.240760 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:57:43.240762 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:57:43.241188 (+ 426us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:57:43.241374 (+ 186us) server_negotiation.cc:1035] Waiting for connection context
0812 06:57:43.241763 (+ 389us) server_negotiation.cc:299] Negotiation successful
0812 06:57:43.242045 (+ 282us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":5,"server-negotiator.queue_time_us":355,"spinlock_wait_cycles":48512,"thread_start_us":207,"threads_started":1}
I20260812 06:57:43.245970 2545 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.19.52.125" port: 34425 } has no permanent_uuid. Determining permanent_uuid...
I20260812 06:57:43.246883 2549 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:57:43.246525 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.125:34425 (local address 127.0.0.1:57844)
0812 06:57:43.246685 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0812 06:57:43.246810 (+ 125us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.52.125:34425: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":77}
W20260812 06:57:43.247143 2491 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.19.52.125:34425: connect: Connection refused (error 111)
I20260812 06:57:43.248154 2549 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:57:43.247904 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.125:34425 (local address 127.0.0.1:57850)
0812 06:57:43.248051 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0812 06:57:43.248097 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.52.125:34425: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":84}
W20260812 06:57:43.248585 2545 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.19.52.125:34425: Network error: Client connection negotiation failed: client connection to 127.19.52.125:34425: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.52.125 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786517687384214-19665-0/krb5kdc/kudu_127.19.52.125.keytab.
Entry for principal kudu/127.19.52.125 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786517687384214-19665-0/krb5kdc/kudu_127.19.52.125.keytab.
I20260812 06:57:43.252969 19665 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu
/tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786517687384214-19665-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786517687384214-19665-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786517687384214-19665-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786517687384214-19665-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.52.125:34425
--webserver_interface=127.19.52.125
--webserver_port=0
--keytab_file=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786517687384214-19665-0/krb5kdc/kudu_127.19.52.125.keytab
--principal=kudu/127.19.52.125
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.52.84:33491
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.19.52.126:46371,127.19.52.125:34425,127.19.52.124:39693
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786517687384214-19665-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786517687384214-19665-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786517687384214-19665-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 06:57:43.297868 2545 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.19.52.125" port: 34425 } attempt: 1
I20260812 06:57:43.299007 2549 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:57:43.298685 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.125:34425 (local address 127.0.0.1:57864)
0812 06:57:43.298858 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0812 06:57:43.298923 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.52.125:34425: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":69}
I20260812 06:57:43.299793 2549 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:57:43.299567 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.125:34425 (local address 127.0.0.1:57878)
0812 06:57:43.299681 (+ 114us) negotiation.cc:107] Waiting for socket to connect
0812 06:57:43.299737 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.52.125:34425: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260812 06:57:43.299932 2545 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.19.52.125:34425: Network error: Client connection negotiation failed: client connection to 127.19.52.125:34425: connect: Connection refused (error 111)
I20260812 06:57:43.368147 2545 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.19.52.125" port: 34425 } attempt: 2
I20260812 06:57:43.369256 2549 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:57:43.368859 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.125:34425 (local address 127.0.0.1:57886)
0812 06:57:43.369093 (+ 234us) negotiation.cc:107] Waiting for socket to connect
0812 06:57:43.369159 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.52.125:34425: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":146}
I20260812 06:57:43.370148 2549 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:57:43.369896 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.125:34425 (local address 127.0.0.1:57888)
0812 06:57:43.370025 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0812 06:57:43.370082 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.52.125:34425: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260812 06:57:43.370293 2545 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.19.52.125:34425: Network error: Client connection negotiation failed: client connection to 127.19.52.125:34425: connect: Connection refused (error 111)
W20260812 06:57:43.374372 2554 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:57:43.374663 2554 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:57:43.374748 2554 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 06:57:43.379251 2554 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 06:57:43.379319 2554 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 06:57:43.379341 2554 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 06:57:43.379359 2554 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 06:57:43.379400 2554 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 06:57:43.385426 2554 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.52.84:33491
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786517687384214-19665-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786517687384214-19665-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.19.52.126:46371,127.19.52.125:34425,127.19.52.124:39693
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786517687384214-19665-0/krb5kdc/kudu_127.19.52.125.keytab
--principal=kudu/127.19.52.125
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.52.125:34425
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786517687384214-19665-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.19.52.125
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2554
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786517687384214-19665-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 06:15:54 UTC on ccycloud.quasar-rnfzce.root.comops.site
build id 4581989
I20260812 06:57:43.386628 2554 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:57:43.387712 2554 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 06:57:43.396857 2562 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:57:43.396862 2559 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:57:43.396947 2554 server_base.cc:1034] running on GCE node
W20260812 06:57:43.396857 2560 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:57:43.397954 2554 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:57:43.399092 2554 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:57:43.400298 2554 hybrid_clock.cc:648] HybridClock initialized: now 1786517863400274 us; error 34 us; skew 500 ppm
Aug 12 06:57:43 dist-test-slave-65mx krb5kdc[2453](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517863, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.52.125@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 06:57:43.403529 2554 init.cc:377] Logged in from keytab as kudu/127.19.52.125@KRBTEST.COM (short username kudu)
I20260812 06:57:43.404680 2554 webserver.cc:460] Webserver started at http://127.19.52.125:34215/ using document root <none> and password file <none>
I20260812 06:57:43.405328 2554 fs_manager.cc:359] Metadata directory not provided
I20260812 06:57:43.405400 2554 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:57:43.405656 2554 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 06:57:43.408298 2554 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786517687384214-19665-0/minicluster-data/master-1/data/instance:
uuid: "fed6a9c81a624eca9e8602638d72313e"
format_stamp: "Formatted at 2026-08-12 06:57:43 on dist-test-slave-65mx"
server_key: "b90a7306fb361520f5b91a45e63f1304"
server_key_iv: "6bf4ff7d64f31ed62a964a9589c514c5"
server_key_version: "encryptionkey@0"
I20260812 06:57:43.409328 2554 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786517687384214-19665-0/minicluster-data/master-1/wal/instance:
uuid: "fed6a9c81a624eca9e8602638d72313e"
format_stamp: "Formatted at 2026-08-12 06:57:43 on dist-test-slave-65mx"
server_key: "b90a7306fb361520f5b91a45e63f1304"
server_key_iv: "6bf4ff7d64f31ed62a964a9589c514c5"
server_key_version: "encryptionkey@0"
I20260812 06:57:43.415256 2554 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.000s
I20260812 06:57:43.418848 2573 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 06:57:43.420192 2554 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260812 06:57:43.420351 2554 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786517687384214-19665-0/minicluster-data/master-1/data,/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786517687384214-19665-0/minicluster-data/master-1/wal
uuid: "fed6a9c81a624eca9e8602638d72313e"
format_stamp: "Formatted at 2026-08-12 06:57:43 on dist-test-slave-65mx"
server_key: "b90a7306fb361520f5b91a45e63f1304"
server_key_iv: "6bf4ff7d64f31ed62a964a9589c514c5"
server_key_version: "encryptionkey@0"
I20260812 06:57:43.420492 2554 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786517687384214-19665-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786517687384214-19665-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786517687384214-19665-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:57:43.446194 2554 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 06:57:43.449996 2554 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 06:57:43.450235 2554 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:57:43.458294 2554 rpc_server.cc:307] RPC server started. Bound to: 127.19.52.125:34425
I20260812 06:57:43.458329 2635 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.52.125:34425 every 8 connection(s)
I20260812 06:57:43.459342 2554 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786517687384214-19665-0/minicluster-data/master-1/data/info.pb
I20260812 06:57:43.461437 2545 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.19.52.125" port: 34425 } attempt: 3
I20260812 06:57:43.461799 19665 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu as pid 2554
I20260812 06:57:43.461967 19665 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786517687384214-19665-0/minicluster-data/master-1/wal/instance
I20260812 06:57:43.462360 19665 external_mini_cluster.cc:1428] Setting key 9320592cd11c3f0adf93306fcc15392e
I20260812 06:57:43.463227 2636 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 06:57:43.467622 2636 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.19.52.126" port: 46371 } has no permanent_uuid. Determining permanent_uuid...
Aug 12 06:57:43 dist-test-slave-65mx krb5kdc[2453](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517863, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.52.126@KRBTEST.COM for kudu/127.19.52.125@KRBTEST.COM
WARNING: no policy specified for kudu/127.19.52.124@KRBTEST.COM; defaulting to no policy
I20260812 06:57:43.476858 2549 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:57:43.462048 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.125:34425 (local address 127.0.0.1:57902)
0812 06:57:43.462181 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0812 06:57:43.462199 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0812 06:57:43.462297 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:57:43.465427 (+ 3130us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:57:43.465430 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:57:43.465645 (+ 215us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:57:43.465786 (+ 141us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:57:43.465794 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:57:43.467081 (+ 1287us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:57:43.467084 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:57:43.467646 (+ 562us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:57:43.467656 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:57:43.468470 (+ 814us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:57:43.468474 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:57:43.468611 (+ 137us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:57:43.469206 (+ 595us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 06:57:43.469228 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:57:43.470829 (+ 1601us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 06:57:43.474470 (+ 3641us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:57:43.474473 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:57:43.474475 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:57:43.474791 (+ 316us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:57:43.475257 (+ 466us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:57:43.475260 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:57:43.475262 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:57:43.475361 (+ 99us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:57:43.476235 (+ 874us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 06:57:43.476237 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 06:57:43.476333 (+ 96us) client_negotiation.cc:769] Sending connection context
0812 06:57:43.476435 (+ 102us) client_negotiation.cc:240] Negotiation successful
0812 06:57:43.476644 (+ 209us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":38}
I20260812 06:57:43.478149 2638 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:57:43.463000 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57902 (local address 127.19.52.125:34425)
0812 06:57:43.463591 (+ 591us) server_negotiation.cc:206] Beginning negotiation
0812 06:57:43.463615 (+ 24us) server_negotiation.cc:399] Waiting for connection header
0812 06:57:43.463664 (+ 49us) server_negotiation.cc:407] Connection header received
0812 06:57:43.464764 (+ 1100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:57:43.464784 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:57:43.465128 (+ 344us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:57:43.465485 (+ 357us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:57:43.465950 (+ 465us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:57:43.466924 (+ 974us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:57:43.467783 (+ 859us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:57:43.468329 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:57:43.468511 (+ 182us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:57:43.471055 (+ 2544us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:57:43.471065 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:57:43.471081 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:57:43.471117 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:57:43.474339 (+ 3222us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:57:43.474931 (+ 592us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:57:43.474941 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:57:43.474951 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:57:43.475123 (+ 172us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:57:43.475496 (+ 373us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:57:43.475502 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:57:43.475505 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:57:43.476108 (+ 603us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:57:43.476234 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0812 06:57:43.476760 (+ 526us) server_negotiation.cc:299] Negotiation successful
0812 06:57:43.477146 (+ 386us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":390,"thread_start_us":164,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.52.124@KRBTEST.COM" created.
Aug 12 06:57:43 dist-test-slave-65mx krb5kdc[2453](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517863, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.52.125@KRBTEST.COM for kudu/127.19.52.126@KRBTEST.COM
I20260812 06:57:43.480679 2545 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.19.52.124" port: 39693 } has no permanent_uuid. Determining permanent_uuid...
I20260812 06:57:43.482097 2549 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:57:43.481249 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.124:39693 (local address 127.0.0.1:57022)
0812 06:57:43.481963 (+ 714us) negotiation.cc:107] Waiting for socket to connect
0812 06:57:43.482015 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.52.124:39693: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":636}
I20260812 06:57:43.483063 2549 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:57:43.482700 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.124:39693 (local address 127.0.0.1:57024)
0812 06:57:43.482912 (+ 212us) negotiation.cc:107] Waiting for socket to connect
0812 06:57:43.482981 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.52.124:39693: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":108}
W20260812 06:57:43.483276 2545 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.19.52.124:39693: Network error: Client connection negotiation failed: client connection to 127.19.52.124:39693: connect: Connection refused (error 111)
I20260812 06:57:43.485172 2640 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:57:43.469667 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.126:46371 (local address 127.0.0.1:40510)
0812 06:57:43.470218 (+ 551us) negotiation.cc:107] Waiting for socket to connect
0812 06:57:43.470281 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0812 06:57:43.470473 (+ 192us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:57:43.470937 (+ 464us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:57:43.470944 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:57:43.471441 (+ 497us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:57:43.471634 (+ 193us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:57:43.471645 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:57:43.472903 (+ 1258us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:57:43.472907 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:57:43.473937 (+ 1030us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:57:43.473949 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:57:43.474685 (+ 736us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:57:43.474690 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:57:43.474825 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:57:43.475391 (+ 566us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 06:57:43.475403 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:57:43.480073 (+ 4670us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 06:57:43.482689 (+ 2616us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:57:43.482704 (+ 15us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:57:43.482711 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:57:43.483040 (+ 329us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:57:43.483603 (+ 563us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:57:43.483605 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:57:43.483607 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:57:43.483709 (+ 102us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:57:43.484270 (+ 561us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 06:57:43.484276 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 06:57:43.484640 (+ 364us) client_negotiation.cc:769] Sending connection context
0812 06:57:43.484822 (+ 182us) client_negotiation.cc:240] Negotiation successful
0812 06:57:43.485013 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":326,"thread_start_us":177,"threads_started":1}
I20260812 06:57:43.485378 2548 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:57:43.469695 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40510 (local address 127.19.52.126:46371)
0812 06:57:43.469884 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0812 06:57:43.469891 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 06:57:43.470366 (+ 475us) server_negotiation.cc:407] Connection header received
0812 06:57:43.470601 (+ 235us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:57:43.470605 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:57:43.470669 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:57:43.470724 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:57:43.471791 (+ 1067us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:57:43.472734 (+ 943us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:57:43.474092 (+ 1358us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:57:43.474534 (+ 442us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:57:43.474675 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:57:43.480334 (+ 5659us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:57:43.480339 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:57:43.480341 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:57:43.480377 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:57:43.482449 (+ 2072us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:57:43.483291 (+ 842us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:57:43.483298 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:57:43.483303 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:57:43.483397 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:57:43.483814 (+ 417us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:57:43.483820 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:57:43.483825 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:57:43.484126 (+ 301us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:57:43.484237 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0812 06:57:43.484957 (+ 720us) server_negotiation.cc:299] Negotiation successful
0812 06:57:43.485143 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83}
I20260812 06:57:43.486743 2636 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.19.52.125" port: 34425 } has no permanent_uuid. Determining permanent_uuid...
Aug 12 06:57:43 dist-test-slave-65mx krb5kdc[2453](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517863, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.52.125@KRBTEST.COM for kudu/127.19.52.125@KRBTEST.COM
I20260812 06:57:43.498632 2638 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:57:43.487449 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57918 (local address 127.19.52.125:34425)
0812 06:57:43.487552 (+ 103us) server_negotiation.cc:206] Beginning negotiation
0812 06:57:43.487557 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 06:57:43.487614 (+ 57us) server_negotiation.cc:407] Connection header received
0812 06:57:43.487744 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:57:43.487748 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:57:43.487786 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:57:43.487874 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:57:43.488274 (+ 400us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:57:43.488874 (+ 600us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:57:43.489634 (+ 760us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:57:43.489857 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:57:43.489969 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:57:43.493999 (+ 4030us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:57:43.494002 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:57:43.494004 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:57:43.494029 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:57:43.496250 (+ 2221us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:57:43.497038 (+ 788us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:57:43.497043 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:57:43.497044 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:57:43.497151 (+ 107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:57:43.497457 (+ 306us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:57:43.497460 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:57:43.497469 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:57:43.497688 (+ 219us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:57:43.497808 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0812 06:57:43.498259 (+ 451us) server_negotiation.cc:299] Negotiation successful
0812 06:57:43.498387 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260812 06:57:43.498761 2640 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:57:43.487399 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.125:34425 (local address 127.0.0.1:57918)
0812 06:57:43.487532 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0812 06:57:43.487548 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0812 06:57:43.487646 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:57:43.487884 (+ 238us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:57:43.487886 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:57:43.488064 (+ 178us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:57:43.488160 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:57:43.488166 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:57:43.488987 (+ 821us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:57:43.488990 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:57:43.489507 (+ 517us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:57:43.489514 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:57:43.489970 (+ 456us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:57:43.489973 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:57:43.490065 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:57:43.490609 (+ 544us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 06:57:43.490641 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:57:43.493785 (+ 3144us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 06:57:43.496530 (+ 2745us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:57:43.496536 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:57:43.496538 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:57:43.496891 (+ 353us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:57:43.497254 (+ 363us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:57:43.497257 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:57:43.497258 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:57:43.497355 (+ 97us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:57:43.497930 (+ 575us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 06:57:43.497938 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 06:57:43.498056 (+ 118us) client_negotiation.cc:769] Sending connection context
0812 06:57:43.498166 (+ 110us) client_negotiation.cc:240] Negotiation successful
0812 06:57:43.498322 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":52}
I20260812 06:57:43.499814 2636 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.19.52.124" port: 39693 } has no permanent_uuid. Determining permanent_uuid...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.52.124 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786517687384214-19665-0/krb5kdc/kudu_127.19.52.124.keytab.
Entry for principal kudu/127.19.52.124 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786517687384214-19665-0/krb5kdc/kudu_127.19.52.124.keytab.
I20260812 06:57:43.500801 2640 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:57:43.500379 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.124:39693 (local address 127.0.0.1:57030)
0812 06:57:43.500640 (+ 261us) negotiation.cc:107] Waiting for socket to connect
0812 06:57:43.500741 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.52.124:39693: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
I20260812 06:57:43.501282 19665 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu
/tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786517687384214-19665-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786517687384214-19665-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786517687384214-19665-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786517687384214-19665-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.52.124:39693
--webserver_interface=127.19.52.124
--webserver_port=0
--keytab_file=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786517687384214-19665-0/krb5kdc/kudu_127.19.52.124.keytab
--principal=kudu/127.19.52.124
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.52.84:33491
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.19.52.126:46371,127.19.52.125:34425,127.19.52.124:39693
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786517687384214-19665-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786517687384214-19665-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786517687384214-19665-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 06:57:43.501653 2574 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.19.52.124:39693: connect: Connection refused (error 111)
I20260812 06:57:43.502712 2640 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:57:43.502460 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.124:39693 (local address 127.0.0.1:57046)
0812 06:57:43.502602 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0812 06:57:43.502654 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.52.124:39693: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":63}
W20260812 06:57:43.502972 2636 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.19.52.124:39693: Network error: Client connection negotiation failed: client connection to 127.19.52.124:39693: connect: Connection refused (error 111)
I20260812 06:57:43.514505 2545 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.19.52.124" port: 39693 } attempt: 1
I20260812 06:57:43.515489 2549 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:57:43.515170 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.124:39693 (local address 127.0.0.1:57048)
0812 06:57:43.515338 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0812 06:57:43.515407 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.52.124:39693: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":69}
I20260812 06:57:43.516322 2549 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:57:43.516099 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.124:39693 (local address 127.0.0.1:57062)
0812 06:57:43.516216 (+ 117us) negotiation.cc:107] Waiting for socket to connect
0812 06:57:43.516268 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.52.124:39693: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
W20260812 06:57:43.516491 2545 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.19.52.124:39693: Network error: Client connection negotiation failed: client connection to 127.19.52.124:39693: connect: Connection refused (error 111)
I20260812 06:57:43.552289 2636 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.19.52.124" port: 39693 } attempt: 1
I20260812 06:57:43.553354 2640 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:57:43.553005 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.124:39693 (local address 127.0.0.1:57078)
0812 06:57:43.553206 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0812 06:57:43.553273 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.52.124:39693: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":100}
I20260812 06:57:43.554376 2640 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:57:43.554086 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.124:39693 (local address 127.0.0.1:57090)
0812 06:57:43.554249 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0812 06:57:43.554311 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.52.124:39693: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":85}
W20260812 06:57:43.554534 2636 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.19.52.124:39693: Network error: Client connection negotiation failed: client connection to 127.19.52.124:39693: connect: Connection refused (error 111)
I20260812 06:57:43.591722 2545 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.19.52.124" port: 39693 } attempt: 2
I20260812 06:57:43.592895 2549 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:57:43.592461 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.124:39693 (local address 127.0.0.1:57096)
0812 06:57:43.592719 (+ 258us) negotiation.cc:107] Waiting for socket to connect
0812 06:57:43.592798 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.52.124:39693: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":116}
I20260812 06:57:43.593909 2549 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:57:43.593579 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.124:39693 (local address 127.0.0.1:57106)
0812 06:57:43.593742 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0812 06:57:43.593810 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.52.124:39693: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":66}
W20260812 06:57:43.594152 2545 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.19.52.124:39693: Network error: Client connection negotiation failed: client connection to 127.19.52.124:39693: connect: Connection refused (error 111)
I20260812 06:57:43.622709 2636 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.19.52.124" port: 39693 } attempt: 2
I20260812 06:57:43.623610 2640 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:57:43.623293 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.124:39693 (local address 127.0.0.1:57110)
0812 06:57:43.623444 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0812 06:57:43.623516 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.52.124:39693: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":63}
I20260812 06:57:43.624430 2640 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:57:43.624216 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.124:39693 (local address 127.0.0.1:57124)
0812 06:57:43.624323 (+ 107us) negotiation.cc:107] Waiting for socket to connect
0812 06:57:43.624375 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.52.124:39693: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
W20260812 06:57:43.624629 2636 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.19.52.124:39693: Network error: Client connection negotiation failed: client connection to 127.19.52.124:39693: connect: Connection refused (error 111)
W20260812 06:57:43.627471 2643 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:57:43.627758 2643 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:57:43.627844 2643 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 06:57:43.632226 2643 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 06:57:43.632316 2643 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 06:57:43.632337 2643 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 06:57:43.632354 2643 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 06:57:43.632370 2643 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 06:57:43.639249 2643 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.52.84:33491
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786517687384214-19665-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786517687384214-19665-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.19.52.126:46371,127.19.52.125:34425,127.19.52.124:39693
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786517687384214-19665-0/krb5kdc/kudu_127.19.52.124.keytab
--principal=kudu/127.19.52.124
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.52.124:39693
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786517687384214-19665-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.19.52.124
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2643
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786517687384214-19665-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 06:15:54 UTC on ccycloud.quasar-rnfzce.root.comops.site
build id 4581989
I20260812 06:57:43.640759 2643 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:57:43.641894 2643 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 06:57:43.651237 2649 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:57:43.651237 2651 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:57:43.651237 2648 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:57:43.651754 2643 server_base.cc:1034] running on GCE node
I20260812 06:57:43.652410 2643 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:57:43.653666 2643 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:57:43.654848 2643 hybrid_clock.cc:648] HybridClock initialized: now 1786517863654825 us; error 40 us; skew 500 ppm
Aug 12 06:57:43 dist-test-slave-65mx krb5kdc[2453](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517863, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.52.124@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 06:57:43.657920 2643 init.cc:377] Logged in from keytab as kudu/127.19.52.124@KRBTEST.COM (short username kudu)
I20260812 06:57:43.658965 2643 webserver.cc:460] Webserver started at http://127.19.52.124:43929/ using document root <none> and password file <none>
I20260812 06:57:43.659552 2643 fs_manager.cc:359] Metadata directory not provided
I20260812 06:57:43.659617 2643 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:57:43.659854 2643 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 06:57:43.662213 2643 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786517687384214-19665-0/minicluster-data/master-2/data/instance:
uuid: "5f78b90dcd374543aaf04248962931d0"
format_stamp: "Formatted at 2026-08-12 06:57:43 on dist-test-slave-65mx"
server_key: "cd665d0b1ca530d27bf840805f96e92a"
server_key_iv: "4a865dc917b66fb07253722082d65a69"
server_key_version: "encryptionkey@0"
I20260812 06:57:43.662811 2643 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786517687384214-19665-0/minicluster-data/master-2/wal/instance:
uuid: "5f78b90dcd374543aaf04248962931d0"
format_stamp: "Formatted at 2026-08-12 06:57:43 on dist-test-slave-65mx"
server_key: "cd665d0b1ca530d27bf840805f96e92a"
server_key_iv: "4a865dc917b66fb07253722082d65a69"
server_key_version: "encryptionkey@0"
I20260812 06:57:43.666932 2643 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260812 06:57:43.669639 2659 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 06:57:43.670958 2643 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 06:57:43.671101 2643 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786517687384214-19665-0/minicluster-data/master-2/data,/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786517687384214-19665-0/minicluster-data/master-2/wal
uuid: "5f78b90dcd374543aaf04248962931d0"
format_stamp: "Formatted at 2026-08-12 06:57:43 on dist-test-slave-65mx"
server_key: "cd665d0b1ca530d27bf840805f96e92a"
server_key_iv: "4a865dc917b66fb07253722082d65a69"
server_key_version: "encryptionkey@0"
I20260812 06:57:43.671262 2643 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786517687384214-19665-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786517687384214-19665-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786517687384214-19665-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:57:43.693373 2545 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.19.52.124" port: 39693 } attempt: 3
I20260812 06:57:43.693569 2643 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 06:57:43.694308 2549 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:57:43.693948 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.124:39693 (local address 127.0.0.1:57140)
0812 06:57:43.694116 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0812 06:57:43.694194 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.52.124:39693: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":70}
I20260812 06:57:43.695164 2549 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:57:43.694904 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.124:39693 (local address 127.0.0.1:57142)
0812 06:57:43.695031 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0812 06:57:43.695094 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.52.124:39693: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
W20260812 06:57:43.695359 2545 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.19.52.124:39693: Network error: Client connection negotiation failed: client connection to 127.19.52.124:39693: connect: Connection refused (error 111)
I20260812 06:57:43.697760 2643 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 06:57:43.697990 2643 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:57:43.706111 2643 rpc_server.cc:307] RPC server started. Bound to: 127.19.52.124:39693
I20260812 06:57:43.706131 2715 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.52.124:39693 every 8 connection(s)
I20260812 06:57:43.707122 2643 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786517687384214-19665-0/minicluster-data/master-2/data/info.pb
I20260812 06:57:43.710319 19665 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu as pid 2643
I20260812 06:57:43.710355 2716 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 06:57:43.710485 19665 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786517687384214-19665-0/minicluster-data/master-2/wal/instance
I20260812 06:57:43.710806 19665 external_mini_cluster.cc:1428] Setting key e74c7721368f1af851d26aaa75bcc300
I20260812 06:57:43.715197 2716 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.19.52.126" port: 46371 } has no permanent_uuid. Determining permanent_uuid...
I20260812 06:57:43.715817 2636 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.19.52.124" port: 39693 } attempt: 3
Aug 12 06:57:43 dist-test-slave-65mx krb5kdc[2453](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517862, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.19.52.124@KRBTEST.COM
Aug 12 06:57:43 dist-test-slave-65mx krb5kdc[2453](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517863, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.52.125@KRBTEST.COM for kudu/127.19.52.124@KRBTEST.COM
Aug 12 06:57:43 dist-test-slave-65mx krb5kdc[2453](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517863, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.52.124@KRBTEST.COM for kudu/127.19.52.126@KRBTEST.COM
I20260812 06:57:43.727484 2640 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:57:43.716314 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.124:39693 (local address 127.0.0.1:57166)
0812 06:57:43.716496 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0812 06:57:43.716509 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 06:57:43.716601 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:57:43.717092 (+ 491us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:57:43.717095 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:57:43.717300 (+ 205us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:57:43.717406 (+ 106us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:57:43.717414 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:57:43.718470 (+ 1056us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:57:43.718477 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:57:43.719275 (+ 798us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:57:43.719288 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:57:43.719987 (+ 699us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:57:43.719993 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:57:43.720123 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:57:43.720731 (+ 608us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 06:57:43.720752 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:57:43.722160 (+ 1408us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 06:57:43.725343 (+ 3183us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:57:43.725346 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:57:43.725349 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:57:43.725608 (+ 259us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:57:43.725990 (+ 382us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:57:43.725993 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:57:43.725996 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:57:43.726090 (+ 94us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:57:43.726917 (+ 827us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 06:57:43.726920 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 06:57:43.727035 (+ 115us) client_negotiation.cc:769] Sending connection context
0812 06:57:43.727121 (+ 86us) client_negotiation.cc:240] Negotiation successful
0812 06:57:43.727304 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":113}
I20260812 06:57:43.728039 2548 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:57:43.717344 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40522 (local address 127.19.52.126:46371)
0812 06:57:43.717467 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0812 06:57:43.717471 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 06:57:43.717814 (+ 343us) server_negotiation.cc:407] Connection header received
0812 06:57:43.718025 (+ 211us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:57:43.718028 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:57:43.718082 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:57:43.718177 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:57:43.719015 (+ 838us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:57:43.719733 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:57:43.720912 (+ 1179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:57:43.721160 (+ 248us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:57:43.721282 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:57:43.724105 (+ 2823us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:57:43.724109 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:57:43.724112 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:57:43.724144 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:57:43.725942 (+ 1798us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:57:43.726471 (+ 529us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:57:43.726475 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:57:43.726476 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:57:43.726577 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:57:43.726917 (+ 340us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:57:43.726937 (+ 20us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:57:43.726939 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:57:43.727163 (+ 224us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:57:43.727229 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0812 06:57:43.727703 (+ 474us) server_negotiation.cc:299] Negotiation successful
0812 06:57:43.727873 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260812 06:57:43.729260 2721 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:57:43.716412 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57166 (local address 127.19.52.124:39693)
0812 06:57:43.716822 (+ 410us) server_negotiation.cc:206] Beginning negotiation
0812 06:57:43.716833 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0812 06:57:43.716847 (+ 14us) server_negotiation.cc:407] Connection header received
0812 06:57:43.716881 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:57:43.716885 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:57:43.716943 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:57:43.717026 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:57:43.717567 (+ 541us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:57:43.718337 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:57:43.719428 (+ 1091us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:57:43.719836 (+ 408us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:57:43.719938 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:57:43.722320 (+ 2382us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:57:43.722327 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:57:43.722342 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:57:43.722367 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:57:43.725217 (+ 2850us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:57:43.725735 (+ 518us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:57:43.725743 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:57:43.725746 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:57:43.725850 (+ 104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:57:43.726202 (+ 352us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:57:43.726205 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:57:43.726207 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:57:43.726785 (+ 578us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:57:43.727813 (+ 1028us) server_negotiation.cc:1035] Waiting for connection context
0812 06:57:43.727891 (+ 78us) server_negotiation.cc:299] Negotiation successful
0812 06:57:43.728072 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":65,"server-negotiator.queue_time_us":325,"spinlock_wait_cycles":18944,"thread_start_us":214,"threads_started":1}
I20260812 06:57:43.729256 2720 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:57:43.712375 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57156 (local address 127.19.52.124:39693)
0812 06:57:43.712848 (+ 473us) server_negotiation.cc:206] Beginning negotiation
0812 06:57:43.712863 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0812 06:57:43.712899 (+ 36us) server_negotiation.cc:407] Connection header received
0812 06:57:43.713769 (+ 870us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:57:43.713788 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:57:43.714141 (+ 353us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:57:43.714494 (+ 353us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:57:43.715336 (+ 842us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:57:43.716740 (+ 1404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:57:43.717947 (+ 1207us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:57:43.718643 (+ 696us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:57:43.718833 (+ 190us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:57:43.722428 (+ 3595us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:57:43.722431 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:57:43.722434 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:57:43.722468 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:57:43.725072 (+ 2604us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:57:43.725662 (+ 590us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:57:43.725672 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:57:43.725681 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:57:43.725840 (+ 159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:57:43.726208 (+ 368us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:57:43.726211 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:57:43.726212 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:57:43.726785 (+ 573us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:57:43.726987 (+ 202us) server_negotiation.cc:1035] Waiting for connection context
0812 06:57:43.727322 (+ 335us) server_negotiation.cc:299] Negotiation successful
0812 06:57:43.728019 (+ 697us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":30,"server-negotiator.queue_time_us":331,"spinlock_wait_cycles":5632,"thread_start_us":157,"threads_started":1}
I20260812 06:57:43.729259 2722 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:57:43.717245 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.126:46371 (local address 127.0.0.1:40522)
0812 06:57:43.717690 (+ 445us) negotiation.cc:107] Waiting for socket to connect
0812 06:57:43.717743 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0812 06:57:43.717903 (+ 160us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:57:43.718226 (+ 323us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:57:43.718231 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:57:43.718703 (+ 472us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:57:43.718858 (+ 155us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:57:43.718866 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:57:43.719875 (+ 1009us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:57:43.719881 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:57:43.720783 (+ 902us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:57:43.720793 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:57:43.721329 (+ 536us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:57:43.721333 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:57:43.721455 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:57:43.721994 (+ 539us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 06:57:43.722010 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:57:43.723888 (+ 1878us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 06:57:43.726037 (+ 2149us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:57:43.726048 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:57:43.726055 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:57:43.726352 (+ 297us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:57:43.726693 (+ 341us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:57:43.726696 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:57:43.726698 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:57:43.726792 (+ 94us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:57:43.727319 (+ 527us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 06:57:43.727326 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 06:57:43.727442 (+ 116us) client_negotiation.cc:769] Sending connection context
0812 06:57:43.728196 (+ 754us) client_negotiation.cc:240] Negotiation successful
0812 06:57:43.728412 (+ 216us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":292,"spinlock_wait_cycles":6144,"thread_start_us":95,"threads_started":1}
I20260812 06:57:43.730865 2716 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.19.52.125" port: 34425 } has no permanent_uuid. Determining permanent_uuid...
I20260812 06:57:43.734241 2636 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fed6a9c81a624eca9e8602638d72313e: Bootstrap starting.
I20260812 06:57:43.737389 2636 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P fed6a9c81a624eca9e8602638d72313e: Neither blocks nor log segments found. Creating new log.
Aug 12 06:57:43 dist-test-slave-65mx krb5kdc[2453](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517862, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.19.52.126@KRBTEST.COM
I20260812 06:57:43.738332 2636 log.cc:824] T 00000000000000000000000000000000 P fed6a9c81a624eca9e8602638d72313e: Log is configured to *not* fsync() on all Append() calls
I20260812 06:57:43.741112 2636 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fed6a9c81a624eca9e8602638d72313e: No bootstrap required, opened a new log
Aug 12 06:57:43 dist-test-slave-65mx krb5kdc[2453](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517863, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.52.124@KRBTEST.COM for kudu/127.19.52.125@KRBTEST.COM
I20260812 06:57:43.743741 2548 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:57:43.732383 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40524 (local address 127.19.52.126:46371)
0812 06:57:43.732507 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0812 06:57:43.732511 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 06:57:43.732527 (+ 16us) server_negotiation.cc:407] Connection header received
0812 06:57:43.732647 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:57:43.732650 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:57:43.732697 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:57:43.732785 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:57:43.733447 (+ 662us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:57:43.734194 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:57:43.735093 (+ 899us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:57:43.735415 (+ 322us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:57:43.735557 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:57:43.738733 (+ 3176us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:57:43.738740 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:57:43.738746 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:57:43.738783 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:57:43.741765 (+ 2982us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:57:43.742387 (+ 622us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:57:43.742393 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:57:43.742395 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:57:43.742492 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:57:43.742817 (+ 325us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:57:43.742821 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:57:43.742823 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:57:43.743036 (+ 213us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:57:43.743309 (+ 273us) server_negotiation.cc:1035] Waiting for connection context
0812 06:57:43.743432 (+ 123us) server_negotiation.cc:299] Negotiation successful
0812 06:57:43.743567 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260812 06:57:43.746155 2636 raft_consensus.cc:348] T 00000000000000000000000000000000 P fed6a9c81a624eca9e8602638d72313e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "54ebea6e8de44f078f2e8b3885cf7237" member_type: VOTER last_known_addr { host: "127.19.52.126" port: 46371 } } peers { permanent_uuid: "fed6a9c81a624eca9e8602638d72313e" member_type: VOTER last_known_addr { host: "127.19.52.125" port: 34425 } } peers { permanent_uuid: "5f78b90dcd374543aaf04248962931d0" member_type: VOTER last_known_addr { host: "127.19.52.124" port: 39693 } }
I20260812 06:57:43.746451 2636 raft_consensus.cc:374] T 00000000000000000000000000000000 P fed6a9c81a624eca9e8602638d72313e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 06:57:43.746523 2636 raft_consensus.cc:729] T 00000000000000000000000000000000 P fed6a9c81a624eca9e8602638d72313e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fed6a9c81a624eca9e8602638d72313e, State: Initialized, Role: FOLLOWER
I20260812 06:57:43.747113 2636 consensus_queue.cc:260] T 00000000000000000000000000000000 P fed6a9c81a624eca9e8602638d72313e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "54ebea6e8de44f078f2e8b3885cf7237" member_type: VOTER last_known_addr { host: "127.19.52.126" port: 46371 } } peers { permanent_uuid: "fed6a9c81a624eca9e8602638d72313e" member_type: VOTER last_known_addr { host: "127.19.52.125" port: 34425 } } peers { permanent_uuid: "5f78b90dcd374543aaf04248962931d0" member_type: VOTER last_known_addr { host: "127.19.52.124" port: 39693 } }
I20260812 06:57:43.747750 2726 sys_catalog.cc:455] T 00000000000000000000000000000000 P fed6a9c81a624eca9e8602638d72313e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "54ebea6e8de44f078f2e8b3885cf7237" member_type: VOTER last_known_addr { host: "127.19.52.126" port: 46371 } } peers { permanent_uuid: "fed6a9c81a624eca9e8602638d72313e" member_type: VOTER last_known_addr { host: "127.19.52.125" port: 34425 } } peers { permanent_uuid: "5f78b90dcd374543aaf04248962931d0" member_type: VOTER last_known_addr { host: "127.19.52.124" port: 39693 } } }
I20260812 06:57:43.748001 2726 sys_catalog.cc:458] T 00000000000000000000000000000000 P fed6a9c81a624eca9e8602638d72313e [sys.catalog]: This master's current role is: FOLLOWER
I20260812 06:57:43.748627 2638 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:57:43.731736 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57934 (local address 127.19.52.125:34425)
0812 06:57:43.738224 (+ 6488us) server_negotiation.cc:206] Beginning negotiation
0812 06:57:43.738232 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 06:57:43.738254 (+ 22us) server_negotiation.cc:407] Connection header received
0812 06:57:43.738306 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:57:43.738310 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:57:43.738367 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:57:43.738583 (+ 216us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:57:43.739368 (+ 785us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:57:43.740112 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:57:43.741213 (+ 1101us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:57:43.741433 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:57:43.741551 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:57:43.744291 (+ 2740us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:57:43.744295 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:57:43.744298 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:57:43.744329 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:57:43.746297 (+ 1968us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:57:43.746927 (+ 630us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:57:43.746931 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:57:43.746932 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:57:43.747032 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:57:43.747369 (+ 337us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:57:43.747375 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:57:43.747379 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:57:43.747738 (+ 359us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:57:43.747819 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0812 06:57:43.748209 (+ 390us) server_negotiation.cc:299] Negotiation successful
0812 06:57:43.748335 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":6364}
I20260812 06:57:43.749076 2636 sys_catalog.cc:564] T 00000000000000000000000000000000 P fed6a9c81a624eca9e8602638d72313e [sys.catalog]: configured and running, proceeding with master startup.
I20260812 06:57:43.749255 2722 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:57:43.731457 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.125:34425 (local address 127.0.0.1:57934)
0812 06:57:43.731729 (+ 272us) negotiation.cc:107] Waiting for socket to connect
0812 06:57:43.731750 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0812 06:57:43.731821 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:57:43.738811 (+ 6990us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:57:43.738815 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:57:43.739043 (+ 228us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:57:43.739208 (+ 165us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:57:43.739220 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:57:43.740261 (+ 1041us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:57:43.740264 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:57:43.741063 (+ 799us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:57:43.741075 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:57:43.741560 (+ 485us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:57:43.741563 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:57:43.741686 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:57:43.742155 (+ 469us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 06:57:43.742171 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:57:43.744126 (+ 1955us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 06:57:43.746488 (+ 2362us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:57:43.746491 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:57:43.746493 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:57:43.746832 (+ 339us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:57:43.747138 (+ 306us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:57:43.747142 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:57:43.747144 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:57:43.747242 (+ 98us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:57:43.747901 (+ 659us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 06:57:43.747904 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 06:57:43.748008 (+ 104us) client_negotiation.cc:769] Sending connection context
0812 06:57:43.748855 (+ 847us) client_negotiation.cc:240] Negotiation successful
0812 06:57:43.749031 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":195}
I20260812 06:57:43.750412 2716 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.19.52.124" port: 39693 } has no permanent_uuid. Determining permanent_uuid...
Aug 12 06:57:43 dist-test-slave-65mx krb5kdc[2453](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517862, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.19.52.125@KRBTEST.COM
W20260812 06:57:43.756145 2737 catalog_manager.cc:1558] T 00000000000000000000000000000000 P fed6a9c81a624eca9e8602638d72313e: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 06:57:43.756258 2737 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
Aug 12 06:57:43 dist-test-slave-65mx krb5kdc[2453](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517863, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.52.124@KRBTEST.COM for kudu/127.19.52.124@KRBTEST.COM
I20260812 06:57:43.759027 2725 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:57:43.745235 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57940 (local address 127.19.52.125:34425)
0812 06:57:43.745860 (+ 625us) server_negotiation.cc:206] Beginning negotiation
0812 06:57:43.745869 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 06:57:43.745880 (+ 11us) server_negotiation.cc:407] Connection header received
0812 06:57:43.745912 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:57:43.745914 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:57:43.745960 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:57:43.746068 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:57:43.746919 (+ 851us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:57:43.747716 (+ 797us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:57:43.748671 (+ 955us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:57:43.749034 (+ 363us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:57:43.749168 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:57:43.752123 (+ 2955us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:57:43.752127 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:57:43.752130 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:57:43.752157 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:57:43.755652 (+ 3495us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:57:43.756498 (+ 846us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:57:43.756509 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:57:43.756511 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:57:43.756683 (+ 172us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:57:43.757082 (+ 399us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:57:43.757087 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:57:43.757089 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:57:43.757462 (+ 373us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:57:43.758624 (+ 1162us) server_negotiation.cc:1035] Waiting for connection context
0812 06:57:43.758705 (+ 81us) server_negotiation.cc:299] Negotiation successful
0812 06:57:43.758844 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":545,"thread_start_us":170,"threads_started":1}
I20260812 06:57:43.754276 2554 master_runner.cc:186] Error getting master registration for 127.19.52.126:46371: OK, instance_id { permanent_uuid: "54ebea6e8de44f078f2e8b3885cf7237" instance_seqno: 1786517863214518 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260812 06:57:43.759512 2554 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260812 06:57:43.761504 2722 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:57:43.750987 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.124:39693 (local address 127.0.0.1:57178)
0812 06:57:43.751380 (+ 393us) negotiation.cc:107] Waiting for socket to connect
0812 06:57:43.751393 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 06:57:43.751479 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:57:43.751854 (+ 375us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:57:43.751860 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:57:43.752054 (+ 194us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:57:43.752189 (+ 135us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:57:43.752197 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:57:43.753697 (+ 1500us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:57:43.753704 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:57:43.754488 (+ 784us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:57:43.754507 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:57:43.755265 (+ 758us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:57:43.755268 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:57:43.755351 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:57:43.755828 (+ 477us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 06:57:43.755844 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:57:43.757459 (+ 1615us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 06:57:43.759493 (+ 2034us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:57:43.759495 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:57:43.759498 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:57:43.759749 (+ 251us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:57:43.760337 (+ 588us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:57:43.760340 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:57:43.760342 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:57:43.760459 (+ 117us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:57:43.760935 (+ 476us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 06:57:43.760937 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 06:57:43.761029 (+ 92us) client_negotiation.cc:769] Sending connection context
0812 06:57:43.761147 (+ 118us) client_negotiation.cc:240] Negotiation successful
0812 06:57:43.761311 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":349}
I20260812 06:57:43.761891 2720 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:57:43.751217 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57178 (local address 127.19.52.124:39693)
0812 06:57:43.751319 (+ 102us) server_negotiation.cc:206] Beginning negotiation
0812 06:57:43.751324 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 06:57:43.751567 (+ 243us) server_negotiation.cc:407] Connection header received
0812 06:57:43.751601 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:57:43.751604 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:57:43.751652 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:57:43.751728 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:57:43.752314 (+ 586us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:57:43.753581 (+ 1267us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:57:43.754668 (+ 1087us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:57:43.755041 (+ 373us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:57:43.755143 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:57:43.757618 (+ 2475us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:57:43.757622 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:57:43.757624 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:57:43.757646 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:57:43.759375 (+ 1729us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:57:43.759869 (+ 494us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:57:43.759873 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:57:43.759875 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:57:43.759965 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:57:43.760577 (+ 612us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:57:43.760580 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:57:43.760582 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:57:43.760818 (+ 236us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:57:43.760936 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0812 06:57:43.761193 (+ 257us) server_negotiation.cc:299] Negotiation successful
0812 06:57:43.761754 (+ 561us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":332,"server-negotiator.queue_time_us":36}
I20260812 06:57:43.763736 19665 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 06:57:43.765606 2716 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5f78b90dcd374543aaf04248962931d0: Bootstrap starting.
I20260812 06:57:43.768752 2716 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5f78b90dcd374543aaf04248962931d0: Neither blocks nor log segments found. Creating new log.
I20260812 06:57:43.769832 2716 log.cc:824] T 00000000000000000000000000000000 P 5f78b90dcd374543aaf04248962931d0: Log is configured to *not* fsync() on all Append() calls
I20260812 06:57:43.772843 2716 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5f78b90dcd374543aaf04248962931d0: No bootstrap required, opened a new log
I20260812 06:57:43.777222 2720 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:57:43.766609 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57188 (local address 127.19.52.124:39693)
0812 06:57:43.766866 (+ 257us) server_negotiation.cc:206] Beginning negotiation
0812 06:57:43.766872 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 06:57:43.767146 (+ 274us) server_negotiation.cc:407] Connection header received
0812 06:57:43.767338 (+ 192us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:57:43.767341 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:57:43.767387 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:57:43.767435 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:57:43.768469 (+ 1034us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:57:43.769264 (+ 795us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:57:43.770565 (+ 1301us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:57:43.770904 (+ 339us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:57:43.771012 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:57:43.772873 (+ 1861us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:57:43.772877 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:57:43.772880 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:57:43.772910 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:57:43.774748 (+ 1838us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:57:43.775290 (+ 542us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:57:43.775294 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:57:43.775296 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:57:43.775388 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:57:43.775703 (+ 315us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:57:43.775706 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:57:43.775708 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:57:43.776006 (+ 298us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:57:43.776142 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0812 06:57:43.776881 (+ 739us) server_negotiation.cc:299] Negotiation successful
0812 06:57:43.777036 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":179}
I20260812 06:57:43.778017 2716 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5f78b90dcd374543aaf04248962931d0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "54ebea6e8de44f078f2e8b3885cf7237" member_type: VOTER last_known_addr { host: "127.19.52.126" port: 46371 } } peers { permanent_uuid: "fed6a9c81a624eca9e8602638d72313e" member_type: VOTER last_known_addr { host: "127.19.52.125" port: 34425 } } peers { permanent_uuid: "5f78b90dcd374543aaf04248962931d0" member_type: VOTER last_known_addr { host: "127.19.52.124" port: 39693 } }
I20260812 06:57:43.778348 2716 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5f78b90dcd374543aaf04248962931d0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 06:57:43.778424 2716 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5f78b90dcd374543aaf04248962931d0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5f78b90dcd374543aaf04248962931d0, State: Initialized, Role: FOLLOWER
I20260812 06:57:43.778975 2716 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5f78b90dcd374543aaf04248962931d0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "54ebea6e8de44f078f2e8b3885cf7237" member_type: VOTER last_known_addr { host: "127.19.52.126" port: 46371 } } peers { permanent_uuid: "fed6a9c81a624eca9e8602638d72313e" member_type: VOTER last_known_addr { host: "127.19.52.125" port: 34425 } } peers { permanent_uuid: "5f78b90dcd374543aaf04248962931d0" member_type: VOTER last_known_addr { host: "127.19.52.124" port: 39693 } }
I20260812 06:57:43.779561 2747 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5f78b90dcd374543aaf04248962931d0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "54ebea6e8de44f078f2e8b3885cf7237" member_type: VOTER last_known_addr { host: "127.19.52.126" port: 46371 } } peers { permanent_uuid: "fed6a9c81a624eca9e8602638d72313e" member_type: VOTER last_known_addr { host: "127.19.52.125" port: 34425 } } peers { permanent_uuid: "5f78b90dcd374543aaf04248962931d0" member_type: VOTER last_known_addr { host: "127.19.52.124" port: 39693 } } }
I20260812 06:57:43.779757 2747 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5f78b90dcd374543aaf04248962931d0 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 06:57:43.780464 2716 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5f78b90dcd374543aaf04248962931d0 [sys.catalog]: configured and running, proceeding with master startup.
W20260812 06:57:43.787626 2761 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 5f78b90dcd374543aaf04248962931d0: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 06:57:43.787720 2761 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 06:57:43.784513 2643 master_runner.cc:186] Error getting master registration for 127.19.52.126:46371: OK, instance_id { permanent_uuid: "54ebea6e8de44f078f2e8b3885cf7237" instance_seqno: 1786517863214518 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260812 06:57:43.788199 2643 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260812 06:57:43.789292 2548 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:57:43.770920 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40526 (local address 127.19.52.126:46371)
0812 06:57:43.771088 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0812 06:57:43.771093 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 06:57:43.778618 (+ 7525us) server_negotiation.cc:407] Connection header received
0812 06:57:43.778801 (+ 183us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:57:43.778804 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:57:43.778859 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:57:43.778910 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:57:43.780654 (+ 1744us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:57:43.781468 (+ 814us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:57:43.782617 (+ 1149us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:57:43.782949 (+ 332us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:57:43.783091 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:57:43.785317 (+ 2226us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:57:43.785321 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:57:43.785324 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:57:43.785371 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:57:43.787176 (+ 1805us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:57:43.787712 (+ 536us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:57:43.787715 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:57:43.787717 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:57:43.787814 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:57:43.788245 (+ 431us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:57:43.788249 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:57:43.788253 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:57:43.788497 (+ 244us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:57:43.788725 (+ 228us) server_negotiation.cc:1035] Waiting for connection context
0812 06:57:43.788980 (+ 255us) server_negotiation.cc:299] Negotiation successful
0812 06:57:43.789110 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260812 06:57:43.791095 2725 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:57:43.766735 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57952 (local address 127.19.52.125:34425)
0812 06:57:43.766887 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0812 06:57:43.766892 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 06:57:43.772873 (+ 5981us) server_negotiation.cc:407] Connection header received
0812 06:57:43.773144 (+ 271us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:57:43.773147 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:57:43.773202 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:57:43.773254 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:57:43.774118 (+ 864us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:57:43.774936 (+ 818us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:57:43.776264 (+ 1328us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:57:43.776692 (+ 428us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:57:43.776772 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:57:43.782952 (+ 6180us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:57:43.782956 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:57:43.782959 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:57:43.782993 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:57:43.789131 (+ 6138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:57:43.789660 (+ 529us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:57:43.789664 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:57:43.789666 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:57:43.789774 (+ 108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:57:43.790106 (+ 332us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:57:43.790110 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:57:43.790112 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:57:43.790370 (+ 258us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:57:43.790553 (+ 183us) server_negotiation.cc:1035] Waiting for connection context
0812 06:57:43.790806 (+ 253us) server_negotiation.cc:299] Negotiation successful
0812 06:57:43.790941 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260812 06:57:43.859593 2545 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.19.52.124" port: 39693 } attempt: 4
Aug 12 06:57:43 dist-test-slave-65mx krb5kdc[2453](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517863, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.52.126@KRBTEST.COM for kudu/127.19.52.124@KRBTEST.COM
I20260812 06:57:43.870194 2549 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:57:43.860146 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.124:39693 (local address 127.0.0.1:57200)
0812 06:57:43.860355 (+ 209us) negotiation.cc:107] Waiting for socket to connect
0812 06:57:43.860371 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0812 06:57:43.860459 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:57:43.860820 (+ 361us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:57:43.860823 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:57:43.861059 (+ 236us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:57:43.861179 (+ 120us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:57:43.861185 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:57:43.862237 (+ 1052us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:57:43.862241 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:57:43.863105 (+ 864us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:57:43.863117 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:57:43.863728 (+ 611us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:57:43.863732 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:57:43.863843 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:57:43.864462 (+ 619us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 06:57:43.864483 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:57:43.866298 (+ 1815us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 06:57:43.868431 (+ 2133us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:57:43.868435 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:57:43.868437 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:57:43.868747 (+ 310us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:57:43.869081 (+ 334us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:57:43.869084 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:57:43.869086 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:57:43.869177 (+ 91us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:57:43.869661 (+ 484us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 06:57:43.869664 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 06:57:43.869767 (+ 103us) client_negotiation.cc:769] Sending connection context
0812 06:57:43.869868 (+ 101us) client_negotiation.cc:240] Negotiation successful
0812 06:57:43.870016 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":126}
I20260812 06:57:43.870971 2720 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:57:43.860269 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57200 (local address 127.19.52.124:39693)
0812 06:57:43.860394 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0812 06:57:43.860399 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 06:57:43.860415 (+ 16us) server_negotiation.cc:407] Connection header received
0812 06:57:43.860600 (+ 185us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:57:43.860604 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:57:43.860656 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:57:43.860744 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:57:43.861319 (+ 575us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:57:43.862093 (+ 774us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:57:43.863245 (+ 1152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:57:43.863594 (+ 349us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:57:43.863707 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:57:43.866462 (+ 2755us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:57:43.866466 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:57:43.866469 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:57:43.866499 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:57:43.868299 (+ 1800us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:57:43.868868 (+ 569us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:57:43.868872 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:57:43.868874 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:57:43.868967 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:57:43.869277 (+ 310us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:57:43.869281 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:57:43.869282 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:57:43.869505 (+ 223us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:57:43.869604 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0812 06:57:43.870671 (+ 1067us) server_negotiation.cc:299] Negotiation successful
0812 06:57:43.870815 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260812 06:57:43.874917 2545 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 54ebea6e8de44f078f2e8b3885cf7237: Bootstrap starting.
I20260812 06:57:43.878021 2545 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 54ebea6e8de44f078f2e8b3885cf7237: Neither blocks nor log segments found. Creating new log.
I20260812 06:57:43.879015 2545 log.cc:824] T 00000000000000000000000000000000 P 54ebea6e8de44f078f2e8b3885cf7237: Log is configured to *not* fsync() on all Append() calls
I20260812 06:57:43.881534 2545 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 54ebea6e8de44f078f2e8b3885cf7237: No bootstrap required, opened a new log
I20260812 06:57:43.886161 2545 raft_consensus.cc:348] T 00000000000000000000000000000000 P 54ebea6e8de44f078f2e8b3885cf7237 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "54ebea6e8de44f078f2e8b3885cf7237" member_type: VOTER last_known_addr { host: "127.19.52.126" port: 46371 } } peers { permanent_uuid: "fed6a9c81a624eca9e8602638d72313e" member_type: VOTER last_known_addr { host: "127.19.52.125" port: 34425 } } peers { permanent_uuid: "5f78b90dcd374543aaf04248962931d0" member_type: VOTER last_known_addr { host: "127.19.52.124" port: 39693 } }
I20260812 06:57:43.886534 2545 raft_consensus.cc:374] T 00000000000000000000000000000000 P 54ebea6e8de44f078f2e8b3885cf7237 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 06:57:43.886641 2545 raft_consensus.cc:729] T 00000000000000000000000000000000 P 54ebea6e8de44f078f2e8b3885cf7237 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 54ebea6e8de44f078f2e8b3885cf7237, State: Initialized, Role: FOLLOWER
I20260812 06:57:43.887235 2545 consensus_queue.cc:260] T 00000000000000000000000000000000 P 54ebea6e8de44f078f2e8b3885cf7237 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "54ebea6e8de44f078f2e8b3885cf7237" member_type: VOTER last_known_addr { host: "127.19.52.126" port: 46371 } } peers { permanent_uuid: "fed6a9c81a624eca9e8602638d72313e" member_type: VOTER last_known_addr { host: "127.19.52.125" port: 34425 } } peers { permanent_uuid: "5f78b90dcd374543aaf04248962931d0" member_type: VOTER last_known_addr { host: "127.19.52.124" port: 39693 } }
I20260812 06:57:43.888799 2545 sys_catalog.cc:564] T 00000000000000000000000000000000 P 54ebea6e8de44f078f2e8b3885cf7237 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 06:57:43.891393 2764 sys_catalog.cc:455] T 00000000000000000000000000000000 P 54ebea6e8de44f078f2e8b3885cf7237 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "54ebea6e8de44f078f2e8b3885cf7237" member_type: VOTER last_known_addr { host: "127.19.52.126" port: 46371 } } peers { permanent_uuid: "fed6a9c81a624eca9e8602638d72313e" member_type: VOTER last_known_addr { host: "127.19.52.125" port: 34425 } } peers { permanent_uuid: "5f78b90dcd374543aaf04248962931d0" member_type: VOTER last_known_addr { host: "127.19.52.124" port: 39693 } } }
I20260812 06:57:43.891544 2764 sys_catalog.cc:458] T 00000000000000000000000000000000 P 54ebea6e8de44f078f2e8b3885cf7237 [sys.catalog]: This master's current role is: FOLLOWER
W20260812 06:57:43.894450 2776 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 54ebea6e8de44f078f2e8b3885cf7237: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 06:57:43.894647 2776 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 06:57:43.897673 2470 master_runner.cc:263] No leader master found from master 54ebea6e8de44f078f2e8b3885cf7237
I20260812 06:57:43.897753 2470 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260812 06:57:43.949925 2665 tablet.cc:2369] T 00000000000000000000000000000000 P 5f78b90dcd374543aaf04248962931d0: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 06:57:43.952126 2580 tablet.cc:2369] T 00000000000000000000000000000000 P fed6a9c81a624eca9e8602638d72313e: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 06:57:43.961196 2494 tablet.cc:2369] T 00000000000000000000000000000000 P 54ebea6e8de44f078f2e8b3885cf7237: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 06:57:44.042309 2726 raft_consensus.cc:482] T 00000000000000000000000000000000 P fed6a9c81a624eca9e8602638d72313e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 06:57:44.042557 2726 raft_consensus.cc:504] T 00000000000000000000000000000000 P fed6a9c81a624eca9e8602638d72313e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "54ebea6e8de44f078f2e8b3885cf7237" member_type: VOTER last_known_addr { host: "127.19.52.126" port: 46371 } } peers { permanent_uuid: "fed6a9c81a624eca9e8602638d72313e" member_type: VOTER last_known_addr { host: "127.19.52.125" port: 34425 } } peers { permanent_uuid: "5f78b90dcd374543aaf04248962931d0" member_type: VOTER last_known_addr { host: "127.19.52.124" port: 39693 } }
I20260812 06:57:44.043709 2726 leader_election.cc:288] T 00000000000000000000000000000000 P fed6a9c81a624eca9e8602638d72313e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 54ebea6e8de44f078f2e8b3885cf7237 (127.19.52.126:46371), 5f78b90dcd374543aaf04248962931d0 (127.19.52.124:39693)
I20260812 06:57:44.044360 2688 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "fed6a9c81a624eca9e8602638d72313e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5f78b90dcd374543aaf04248962931d0" is_pre_election: true
I20260812 06:57:44.044317 2517 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "fed6a9c81a624eca9e8602638d72313e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "54ebea6e8de44f078f2e8b3885cf7237" is_pre_election: true
I20260812 06:57:44.044782 2688 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 5f78b90dcd374543aaf04248962931d0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fed6a9c81a624eca9e8602638d72313e in term 0.
I20260812 06:57:44.044904 2517 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 54ebea6e8de44f078f2e8b3885cf7237 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fed6a9c81a624eca9e8602638d72313e in term 0.
I20260812 06:57:44.045310 2574 leader_election.cc:302] T 00000000000000000000000000000000 P fed6a9c81a624eca9e8602638d72313e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 54ebea6e8de44f078f2e8b3885cf7237, 5f78b90dcd374543aaf04248962931d0, fed6a9c81a624eca9e8602638d72313e; no voters:
I20260812 06:57:44.045665 2726 raft_consensus.cc:2781] T 00000000000000000000000000000000 P fed6a9c81a624eca9e8602638d72313e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 06:57:44.045774 2726 raft_consensus.cc:482] T 00000000000000000000000000000000 P fed6a9c81a624eca9e8602638d72313e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 06:57:44.045816 2726 raft_consensus.cc:3037] T 00000000000000000000000000000000 P fed6a9c81a624eca9e8602638d72313e [term 0 FOLLOWER]: Advancing to term 1
I20260812 06:57:44.047246 2726 raft_consensus.cc:504] T 00000000000000000000000000000000 P fed6a9c81a624eca9e8602638d72313e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "54ebea6e8de44f078f2e8b3885cf7237" member_type: VOTER last_known_addr { host: "127.19.52.126" port: 46371 } } peers { permanent_uuid: "fed6a9c81a624eca9e8602638d72313e" member_type: VOTER last_known_addr { host: "127.19.52.125" port: 34425 } } peers { permanent_uuid: "5f78b90dcd374543aaf04248962931d0" member_type: VOTER last_known_addr { host: "127.19.52.124" port: 39693 } }
I20260812 06:57:44.047647 2726 leader_election.cc:288] T 00000000000000000000000000000000 P fed6a9c81a624eca9e8602638d72313e [CANDIDATE]: Term 1 election: Requested vote from peers 54ebea6e8de44f078f2e8b3885cf7237 (127.19.52.126:46371), 5f78b90dcd374543aaf04248962931d0 (127.19.52.124:39693)
I20260812 06:57:44.048043 2517 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "fed6a9c81a624eca9e8602638d72313e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "54ebea6e8de44f078f2e8b3885cf7237"
I20260812 06:57:44.048163 2688 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "fed6a9c81a624eca9e8602638d72313e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5f78b90dcd374543aaf04248962931d0"
I20260812 06:57:44.048213 2517 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 54ebea6e8de44f078f2e8b3885cf7237 [term 0 FOLLOWER]: Advancing to term 1
I20260812 06:57:44.048318 2688 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5f78b90dcd374543aaf04248962931d0 [term 0 FOLLOWER]: Advancing to term 1
I20260812 06:57:44.049490 2517 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 54ebea6e8de44f078f2e8b3885cf7237 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fed6a9c81a624eca9e8602638d72313e in term 1.
I20260812 06:57:44.049722 2688 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 5f78b90dcd374543aaf04248962931d0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fed6a9c81a624eca9e8602638d72313e in term 1.
I20260812 06:57:44.049862 2575 leader_election.cc:302] T 00000000000000000000000000000000 P fed6a9c81a624eca9e8602638d72313e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 54ebea6e8de44f078f2e8b3885cf7237, fed6a9c81a624eca9e8602638d72313e; no voters:
I20260812 06:57:44.050040 2726 raft_consensus.cc:2781] T 00000000000000000000000000000000 P fed6a9c81a624eca9e8602638d72313e [term 1 FOLLOWER]: Leader election won for term 1
I20260812 06:57:44.050287 2726 raft_consensus.cc:686] T 00000000000000000000000000000000 P fed6a9c81a624eca9e8602638d72313e [term 1 LEADER]: Becoming Leader. State: Replica: fed6a9c81a624eca9e8602638d72313e, State: Running, Role: LEADER
I20260812 06:57:44.050674 2726 consensus_queue.cc:237] T 00000000000000000000000000000000 P fed6a9c81a624eca9e8602638d72313e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "54ebea6e8de44f078f2e8b3885cf7237" member_type: VOTER last_known_addr { host: "127.19.52.126" port: 46371 } } peers { permanent_uuid: "fed6a9c81a624eca9e8602638d72313e" member_type: VOTER last_known_addr { host: "127.19.52.125" port: 34425 } } peers { permanent_uuid: "5f78b90dcd374543aaf04248962931d0" member_type: VOTER last_known_addr { host: "127.19.52.124" port: 39693 } }
I20260812 06:57:44.052819 2778 sys_catalog.cc:455] T 00000000000000000000000000000000 P fed6a9c81a624eca9e8602638d72313e [sys.catalog]: SysCatalogTable state changed. Reason: New leader fed6a9c81a624eca9e8602638d72313e. Latest consensus state: current_term: 1 leader_uuid: "fed6a9c81a624eca9e8602638d72313e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "54ebea6e8de44f078f2e8b3885cf7237" member_type: VOTER last_known_addr { host: "127.19.52.126" port: 46371 } } peers { permanent_uuid: "fed6a9c81a624eca9e8602638d72313e" member_type: VOTER last_known_addr { host: "127.19.52.125" port: 34425 } } peers { permanent_uuid: "5f78b90dcd374543aaf04248962931d0" member_type: VOTER last_known_addr { host: "127.19.52.124" port: 39693 } } }
I20260812 06:57:44.052948 2778 sys_catalog.cc:458] T 00000000000000000000000000000000 P fed6a9c81a624eca9e8602638d72313e [sys.catalog]: This master's current role is: LEADER
I20260812 06:57:44.053346 2780 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 06:57:44.054438 2780 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 06:57:44.060127 2517 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 54ebea6e8de44f078f2e8b3885cf7237 [term 1 FOLLOWER]: Refusing update from remote peer fed6a9c81a624eca9e8602638d72313e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 06:57:44.060148 2688 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 5f78b90dcd374543aaf04248962931d0 [term 1 FOLLOWER]: Refusing update from remote peer fed6a9c81a624eca9e8602638d72313e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 06:57:44.060887 2726 consensus_queue.cc:1035] T 00000000000000000000000000000000 P fed6a9c81a624eca9e8602638d72313e [LEADER]: Connected to new peer: Peer: permanent_uuid: "5f78b90dcd374543aaf04248962931d0" member_type: VOTER last_known_addr { host: "127.19.52.124" port: 39693 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 06:57:44.061167 2778 consensus_queue.cc:1035] T 00000000000000000000000000000000 P fed6a9c81a624eca9e8602638d72313e [LEADER]: Connected to new peer: Peer: permanent_uuid: "54ebea6e8de44f078f2e8b3885cf7237" member_type: VOTER last_known_addr { host: "127.19.52.126" port: 46371 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 06:57:44.065721 2764 sys_catalog.cc:455] T 00000000000000000000000000000000 P 54ebea6e8de44f078f2e8b3885cf7237 [sys.catalog]: SysCatalogTable state changed. Reason: New leader fed6a9c81a624eca9e8602638d72313e. Latest consensus state: current_term: 1 leader_uuid: "fed6a9c81a624eca9e8602638d72313e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "54ebea6e8de44f078f2e8b3885cf7237" member_type: VOTER last_known_addr { host: "127.19.52.126" port: 46371 } } peers { permanent_uuid: "fed6a9c81a624eca9e8602638d72313e" member_type: VOTER last_known_addr { host: "127.19.52.125" port: 34425 } } peers { permanent_uuid: "5f78b90dcd374543aaf04248962931d0" member_type: VOTER last_known_addr { host: "127.19.52.124" port: 39693 } } }
I20260812 06:57:44.065713 2747 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5f78b90dcd374543aaf04248962931d0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader fed6a9c81a624eca9e8602638d72313e. Latest consensus state: current_term: 1 leader_uuid: "fed6a9c81a624eca9e8602638d72313e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "54ebea6e8de44f078f2e8b3885cf7237" member_type: VOTER last_known_addr { host: "127.19.52.126" port: 46371 } } peers { permanent_uuid: "fed6a9c81a624eca9e8602638d72313e" member_type: VOTER last_known_addr { host: "127.19.52.125" port: 34425 } } peers { permanent_uuid: "5f78b90dcd374543aaf04248962931d0" member_type: VOTER last_known_addr { host: "127.19.52.124" port: 39693 } } }
I20260812 06:57:44.065873 2747 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5f78b90dcd374543aaf04248962931d0 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 06:57:44.065876 2764 sys_catalog.cc:458] T 00000000000000000000000000000000 P 54ebea6e8de44f078f2e8b3885cf7237 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 06:57:44.068989 2778 sys_catalog.cc:455] T 00000000000000000000000000000000 P fed6a9c81a624eca9e8602638d72313e [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "fed6a9c81a624eca9e8602638d72313e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "54ebea6e8de44f078f2e8b3885cf7237" member_type: VOTER last_known_addr { host: "127.19.52.126" port: 46371 } } peers { permanent_uuid: "fed6a9c81a624eca9e8602638d72313e" member_type: VOTER last_known_addr { host: "127.19.52.125" port: 34425 } } peers { permanent_uuid: "5f78b90dcd374543aaf04248962931d0" member_type: VOTER last_known_addr { host: "127.19.52.124" port: 39693 } } }
I20260812 06:57:44.069120 2778 sys_catalog.cc:458] T 00000000000000000000000000000000 P fed6a9c81a624eca9e8602638d72313e [sys.catalog]: This master's current role is: LEADER
I20260812 06:57:44.069591 2747 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5f78b90dcd374543aaf04248962931d0 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "fed6a9c81a624eca9e8602638d72313e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "54ebea6e8de44f078f2e8b3885cf7237" member_type: VOTER last_known_addr { host: "127.19.52.126" port: 46371 } } peers { permanent_uuid: "fed6a9c81a624eca9e8602638d72313e" member_type: VOTER last_known_addr { host: "127.19.52.125" port: 34425 } } peers { permanent_uuid: "5f78b90dcd374543aaf04248962931d0" member_type: VOTER last_known_addr { host: "127.19.52.124" port: 39693 } } }
I20260812 06:57:44.069736 2747 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5f78b90dcd374543aaf04248962931d0 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 06:57:44.070984 2784 mvcc.cc:205] Tried to move back new op lower bound from 7317577171182702592 to 7317577171154595840. Current Snapshot: MvccSnapshot[applied={T|T < 7317577171182702592}]
I20260812 06:57:44.070991 2726 sys_catalog.cc:455] T 00000000000000000000000000000000 P fed6a9c81a624eca9e8602638d72313e [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "fed6a9c81a624eca9e8602638d72313e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "54ebea6e8de44f078f2e8b3885cf7237" member_type: VOTER last_known_addr { host: "127.19.52.126" port: 46371 } } peers { permanent_uuid: "fed6a9c81a624eca9e8602638d72313e" member_type: VOTER last_known_addr { host: "127.19.52.125" port: 34425 } } peers { permanent_uuid: "5f78b90dcd374543aaf04248962931d0" member_type: VOTER last_known_addr { host: "127.19.52.124" port: 39693 } } }
I20260812 06:57:44.071136 2726 sys_catalog.cc:458] T 00000000000000000000000000000000 P fed6a9c81a624eca9e8602638d72313e [sys.catalog]: This master's current role is: LEADER
I20260812 06:57:44.072033 2764 sys_catalog.cc:455] T 00000000000000000000000000000000 P 54ebea6e8de44f078f2e8b3885cf7237 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "fed6a9c81a624eca9e8602638d72313e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "54ebea6e8de44f078f2e8b3885cf7237" member_type: VOTER last_known_addr { host: "127.19.52.126" port: 46371 } } peers { permanent_uuid: "fed6a9c81a624eca9e8602638d72313e" member_type: VOTER last_known_addr { host: "127.19.52.125" port: 34425 } } peers { permanent_uuid: "5f78b90dcd374543aaf04248962931d0" member_type: VOTER last_known_addr { host: "127.19.52.124" port: 39693 } } }
I20260812 06:57:44.072165 2764 sys_catalog.cc:458] T 00000000000000000000000000000000 P 54ebea6e8de44f078f2e8b3885cf7237 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 06:57:44.072845 2780 catalog_manager.cc:1347] Generated new cluster ID: c4ffc670657b464289050a4a752d2362
I20260812 06:57:44.072919 2780 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 06:57:44.102531 2780 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 06:57:44.103852 2780 catalog_manager.cc:1504] Loading token signing keys...
I20260812 06:57:44.117457 2780 catalog_manager.cc:5868] T 00000000000000000000000000000000 P fed6a9c81a624eca9e8602638d72313e: Generated new TSK 0
I20260812 06:57:44.118559 2780 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 06:57:44.789196 2761 catalog_manager.cc:1259] Loaded cluster ID: c4ffc670657b464289050a4a752d2362
I20260812 06:57:44.789317 2761 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 5f78b90dcd374543aaf04248962931d0: loading cluster ID for follower catalog manager: success
I20260812 06:57:44.791113 2761 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 5f78b90dcd374543aaf04248962931d0: acquiring CA information for follower catalog manager: success
I20260812 06:57:44.792384 2761 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 5f78b90dcd374543aaf04248962931d0: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260812 06:57:44.896147 2776 catalog_manager.cc:1259] Loaded cluster ID: c4ffc670657b464289050a4a752d2362
I20260812 06:57:44.896271 2776 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 54ebea6e8de44f078f2e8b3885cf7237: loading cluster ID for follower catalog manager: success
I20260812 06:57:44.898108 2776 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 54ebea6e8de44f078f2e8b3885cf7237: acquiring CA information for follower catalog manager: success
I20260812 06:57:44.899420 2776 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 54ebea6e8de44f078f2e8b3885cf7237: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260812 06:57:45.183531 19665 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu with pid 2470
I20260812 06:57:45.192461 19665 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu with pid 2554
I20260812 06:57:45.201122 19665 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu with pid 2643
2026-08-12T06:57:45Z chronyd exiting
[ OK ] SecurityITest.IPKICACert (3491 ms)
[ RUN ] SecurityITest.IPKICACertWebServerEndPoints
Loading random data
Initializing database '/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786517687384214-19665-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 06:57:45 dist-test-slave-65mx krb5kdc[2796](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 06:57:45 dist-test-slave-65mx krb5kdc[2796](info): set up 2 sockets
Aug 12 06:57:45 dist-test-slave-65mx krb5kdc[2796](info): commencing operation
krb5kdc: starting...
W20260812 06:57:46.335494 19665 mini_kdc.cc:121] Time spent starting KDC: real 1.109s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786517687384214-19665-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786517687384214-19665-0/krb5kdc/test-user.keytab.
Aug 12 06:57:46 dist-test-slave-65mx krb5kdc[2796](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517866, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T06:57:46Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T06:57:46Z Disabled control of system clock
WARNING: no policy specified for kudu/127.19.52.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.52.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.52.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786517687384214-19665-0/krb5kdc/kudu_127.19.52.126.keytab.
Entry for principal kudu/127.19.52.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786517687384214-19665-0/krb5kdc/kudu_127.19.52.126.keytab.
I20260812 06:57:46.486600 19665 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu
/tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786517687384214-19665-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786517687384214-19665-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786517687384214-19665-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786517687384214-19665-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.52.126:41069
--webserver_interface=127.19.52.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786517687384214-19665-0/krb5kdc/kudu_127.19.52.126.keytab
--principal=kudu/127.19.52.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.52.84:38291
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.19.52.126:41069
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786517687384214-19665-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786517687384214-19665-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786517687384214-19665-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 06:57:46.612819 2813 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:57:46.613140 2813 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:57:46.613188 2813 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 06:57:46.617887 2813 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 06:57:46.617985 2813 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 06:57:46.618013 2813 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 06:57:46.618032 2813 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 06:57:46.618048 2813 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 06:57:46.623821 2813 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.52.84:38291
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786517687384214-19665-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786517687384214-19665-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.19.52.126:41069
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786517687384214-19665-0/krb5kdc/kudu_127.19.52.126.keytab
--principal=kudu/127.19.52.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.52.126:41069
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786517687384214-19665-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.19.52.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2813
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786517687384214-19665-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 06:15:54 UTC on ccycloud.quasar-rnfzce.root.comops.site
build id 4581989
I20260812 06:57:46.625134 2813 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:57:46.626175 2813 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260812 06:57:46.635450 2813 server_base.cc:1034] running on GCE node
W20260812 06:57:46.635406 2821 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:57:46.635402 2819 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:57:46.635380 2818 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:57:46.636338 2813 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:57:46.637710 2813 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:57:46.638932 2813 hybrid_clock.cc:648] HybridClock initialized: now 1786517866638911 us; error 38 us; skew 500 ppm
Aug 12 06:57:46 dist-test-slave-65mx krb5kdc[2796](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517866, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.52.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 06:57:46.648988 2813 init.cc:377] Logged in from keytab as kudu/127.19.52.126@KRBTEST.COM (short username kudu)
I20260812 06:57:46.650792 2813 webserver.cc:460] Webserver started at http://127.19.52.126:35483/ using document root <none> and password file <none>
I20260812 06:57:46.651614 2813 fs_manager.cc:359] Metadata directory not provided
I20260812 06:57:46.651778 2813 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:57:46.652043 2813 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 06:57:46.654816 2813 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786517687384214-19665-0/minicluster-data/master-0/data/instance:
uuid: "bca0b269ef6649e4a62b0c8a98d8b085"
format_stamp: "Formatted at 2026-08-12 06:57:46 on dist-test-slave-65mx"
server_key: "b53e1d8208490a3f77fa36298ddb6fe4"
server_key_iv: "4987030736300f9bfa20fdf9b58644bb"
server_key_version: "encryptionkey@0"
I20260812 06:57:46.655567 2813 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786517687384214-19665-0/minicluster-data/master-0/wal/instance:
uuid: "bca0b269ef6649e4a62b0c8a98d8b085"
format_stamp: "Formatted at 2026-08-12 06:57:46 on dist-test-slave-65mx"
server_key: "b53e1d8208490a3f77fa36298ddb6fe4"
server_key_iv: "4987030736300f9bfa20fdf9b58644bb"
server_key_version: "encryptionkey@0"
I20260812 06:57:46.659998 2813 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.004s
I20260812 06:57:46.662978 2831 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 06:57:46.664705 2813 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 06:57:46.664885 2813 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786517687384214-19665-0/minicluster-data/master-0/data,/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786517687384214-19665-0/minicluster-data/master-0/wal
uuid: "bca0b269ef6649e4a62b0c8a98d8b085"
format_stamp: "Formatted at 2026-08-12 06:57:46 on dist-test-slave-65mx"
server_key: "b53e1d8208490a3f77fa36298ddb6fe4"
server_key_iv: "4987030736300f9bfa20fdf9b58644bb"
server_key_version: "encryptionkey@0"
I20260812 06:57:46.665035 2813 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786517687384214-19665-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786517687384214-19665-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786517687384214-19665-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:57:46.695567 2813 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 06:57:46.699153 2813 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 06:57:46.699383 2813 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:57:46.707206 2813 rpc_server.cc:307] RPC server started. Bound to: 127.19.52.126:41069
I20260812 06:57:46.707231 2889 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.52.126:41069 every 8 connection(s)
I20260812 06:57:46.708220 2813 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786517687384214-19665-0/minicluster-data/master-0/data/info.pb
I20260812 06:57:46.711133 2890 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 06:57:46.716290 19665 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu as pid 2813
I20260812 06:57:46.716454 19665 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786517687384214-19665-0/minicluster-data/master-0/wal/instance
I20260812 06:57:46.716825 19665 external_mini_cluster.cc:1428] Setting key 9f1437a8226320155dd01c03a7f145ce
I20260812 06:57:46.717764 2890 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bca0b269ef6649e4a62b0c8a98d8b085: Bootstrap starting.
I20260812 06:57:46.720305 2890 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P bca0b269ef6649e4a62b0c8a98d8b085: Neither blocks nor log segments found. Creating new log.
I20260812 06:57:46.721155 2890 log.cc:824] T 00000000000000000000000000000000 P bca0b269ef6649e4a62b0c8a98d8b085: Log is configured to *not* fsync() on all Append() calls
I20260812 06:57:46.723152 2890 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bca0b269ef6649e4a62b0c8a98d8b085: No bootstrap required, opened a new log
Aug 12 06:57:46 dist-test-slave-65mx krb5kdc[2796](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517866, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.19.52.126@KRBTEST.COM
I20260812 06:57:46.727430 2890 raft_consensus.cc:348] T 00000000000000000000000000000000 P bca0b269ef6649e4a62b0c8a98d8b085 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bca0b269ef6649e4a62b0c8a98d8b085" member_type: VOTER last_known_addr { host: "127.19.52.126" port: 41069 } }
I20260812 06:57:46.727746 2890 raft_consensus.cc:374] T 00000000000000000000000000000000 P bca0b269ef6649e4a62b0c8a98d8b085 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 06:57:46.727792 2890 raft_consensus.cc:729] T 00000000000000000000000000000000 P bca0b269ef6649e4a62b0c8a98d8b085 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bca0b269ef6649e4a62b0c8a98d8b085, State: Initialized, Role: FOLLOWER
I20260812 06:57:46.728312 2890 consensus_queue.cc:260] T 00000000000000000000000000000000 P bca0b269ef6649e4a62b0c8a98d8b085 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bca0b269ef6649e4a62b0c8a98d8b085" member_type: VOTER last_known_addr { host: "127.19.52.126" port: 41069 } }
I20260812 06:57:46.728443 2890 raft_consensus.cc:388] T 00000000000000000000000000000000 P bca0b269ef6649e4a62b0c8a98d8b085 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 06:57:46.728487 2890 raft_consensus.cc:482] T 00000000000000000000000000000000 P bca0b269ef6649e4a62b0c8a98d8b085 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 06:57:46.728618 2890 raft_consensus.cc:3037] T 00000000000000000000000000000000 P bca0b269ef6649e4a62b0c8a98d8b085 [term 0 FOLLOWER]: Advancing to term 1
I20260812 06:57:46.729681 2890 raft_consensus.cc:504] T 00000000000000000000000000000000 P bca0b269ef6649e4a62b0c8a98d8b085 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bca0b269ef6649e4a62b0c8a98d8b085" member_type: VOTER last_known_addr { host: "127.19.52.126" port: 41069 } }
I20260812 06:57:46.730072 2890 leader_election.cc:302] T 00000000000000000000000000000000 P bca0b269ef6649e4a62b0c8a98d8b085 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: bca0b269ef6649e4a62b0c8a98d8b085; no voters:
I20260812 06:57:46.730531 2890 leader_election.cc:288] T 00000000000000000000000000000000 P bca0b269ef6649e4a62b0c8a98d8b085 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 06:57:46.730616 2897 raft_consensus.cc:2781] T 00000000000000000000000000000000 P bca0b269ef6649e4a62b0c8a98d8b085 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 06:57:46.730834 2897 raft_consensus.cc:686] T 00000000000000000000000000000000 P bca0b269ef6649e4a62b0c8a98d8b085 [term 1 LEADER]: Becoming Leader. State: Replica: bca0b269ef6649e4a62b0c8a98d8b085, State: Running, Role: LEADER
I20260812 06:57:46.731168 2897 consensus_queue.cc:237] T 00000000000000000000000000000000 P bca0b269ef6649e4a62b0c8a98d8b085 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bca0b269ef6649e4a62b0c8a98d8b085" member_type: VOTER last_known_addr { host: "127.19.52.126" port: 41069 } }
I20260812 06:57:46.731986 2890 sys_catalog.cc:564] T 00000000000000000000000000000000 P bca0b269ef6649e4a62b0c8a98d8b085 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 06:57:46.733069 2898 sys_catalog.cc:455] T 00000000000000000000000000000000 P bca0b269ef6649e4a62b0c8a98d8b085 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "bca0b269ef6649e4a62b0c8a98d8b085" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bca0b269ef6649e4a62b0c8a98d8b085" member_type: VOTER last_known_addr { host: "127.19.52.126" port: 41069 } } }
I20260812 06:57:46.733124 2899 sys_catalog.cc:455] T 00000000000000000000000000000000 P bca0b269ef6649e4a62b0c8a98d8b085 [sys.catalog]: SysCatalogTable state changed. Reason: New leader bca0b269ef6649e4a62b0c8a98d8b085. Latest consensus state: current_term: 1 leader_uuid: "bca0b269ef6649e4a62b0c8a98d8b085" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bca0b269ef6649e4a62b0c8a98d8b085" member_type: VOTER last_known_addr { host: "127.19.52.126" port: 41069 } } }
I20260812 06:57:46.733250 2899 sys_catalog.cc:458] T 00000000000000000000000000000000 P bca0b269ef6649e4a62b0c8a98d8b085 [sys.catalog]: This master's current role is: LEADER
I20260812 06:57:46.733201 2898 sys_catalog.cc:458] T 00000000000000000000000000000000 P bca0b269ef6649e4a62b0c8a98d8b085 [sys.catalog]: This master's current role is: LEADER
I20260812 06:57:46.733610 2895 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:57:46.718364 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39410 (local address 127.19.52.126:41069)
0812 06:57:46.718840 (+ 476us) server_negotiation.cc:206] Beginning negotiation
0812 06:57:46.718863 (+ 23us) server_negotiation.cc:399] Waiting for connection header
0812 06:57:46.718900 (+ 37us) server_negotiation.cc:407] Connection header received
0812 06:57:46.719606 (+ 706us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:57:46.719623 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:57:46.719949 (+ 326us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:57:46.720294 (+ 345us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:57:46.721211 (+ 917us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:57:46.722252 (+ 1041us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:57:46.723146 (+ 894us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:57:46.723701 (+ 555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:57:46.723877 (+ 176us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:57:46.726904 (+ 3027us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:57:46.726910 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:57:46.726925 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:57:46.726955 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:57:46.729393 (+ 2438us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:57:46.729948 (+ 555us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:57:46.729954 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:57:46.729959 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:57:46.730096 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:57:46.730407 (+ 311us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:57:46.730410 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:57:46.730411 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:57:46.730986 (+ 575us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:57:46.731864 (+ 878us) server_negotiation.cc:1035] Waiting for connection context
0812 06:57:46.732038 (+ 174us) server_negotiation.cc:299] Negotiation successful
0812 06:57:46.732386 (+ 348us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":330,"thread_start_us":147,"threads_started":1}
I20260812 06:57:46.733661 2908 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 06:57:46.737566 19665 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 06:57:46.738243 2908 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 06:57:46.745321 2908 catalog_manager.cc:1347] Generated new cluster ID: d6ee684838054b39968fc80b1069fbc0
I20260812 06:57:46.745425 2908 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 06:57:46.761026 2908 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 06:57:46.762262 2908 catalog_manager.cc:1504] Loading token signing keys...
I20260812 06:57:46.774070 2908 catalog_manager.cc:5868] T 00000000000000000000000000000000 P bca0b269ef6649e4a62b0c8a98d8b085: Generated new TSK 0
I20260812 06:57:46.774981 2908 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 06:57:46.794684 19665 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu with pid 2813
2026-08-12T06:57:46Z chronyd exiting
[ OK ] SecurityITest.IPKICACertWebServerEndPoints (1591 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedIPKICert
Loading random data
Initializing database '/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517687384214-19665-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 06:57:46 dist-test-slave-65mx krb5kdc[2923](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 06:57:46 dist-test-slave-65mx krb5kdc[2923](info): set up 2 sockets
Aug 12 06:57:46 dist-test-slave-65mx krb5kdc[2923](info): commencing operation
krb5kdc: starting...
W20260812 06:57:47.961766 19665 mini_kdc.cc:121] Time spent starting KDC: real 1.143s user 0.004s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517687384214-19665-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517687384214-19665-0/krb5kdc/test-user.keytab.
Aug 12 06:57:48 dist-test-slave-65mx krb5kdc[2923](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517868, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T06:57:48Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T06:57:48Z Disabled control of system clock
WARNING: no policy specified for kudu/127.19.52.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.52.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.52.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517687384214-19665-0/krb5kdc/kudu_127.19.52.126.keytab.
Entry for principal kudu/127.19.52.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517687384214-19665-0/krb5kdc/kudu_127.19.52.126.keytab.
I20260812 06:57:48.104408 19665 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu
/tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517687384214-19665-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517687384214-19665-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517687384214-19665-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517687384214-19665-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.52.126:42629
--webserver_interface=127.19.52.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517687384214-19665-0/krb5kdc/kudu_127.19.52.126.keytab
--principal=kudu/127.19.52.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.52.84:33591
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.19.52.126:42629
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517687384214-19665-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517687384214-19665-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517687384214-19665-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 06:57:48.227195 2938 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:57:48.227527 2938 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:57:48.227609 2938 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 06:57:48.232587 2938 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 06:57:48.232661 2938 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 06:57:48.232684 2938 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 06:57:48.232704 2938 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 06:57:48.232765 2938 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 06:57:48.239444 2938 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.52.84:33591
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517687384214-19665-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517687384214-19665-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.19.52.126:42629
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517687384214-19665-0/krb5kdc/kudu_127.19.52.126.keytab
--principal=kudu/127.19.52.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.52.126:42629
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517687384214-19665-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.19.52.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2938
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517687384214-19665-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 06:15:54 UTC on ccycloud.quasar-rnfzce.root.comops.site
build id 4581989
I20260812 06:57:48.240813 2938 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:57:48.242040 2938 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 06:57:48.251730 2944 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:57:48.251763 2947 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:57:48.251763 2945 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:57:48.252365 2938 server_base.cc:1034] running on GCE node
I20260812 06:57:48.252961 2938 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:57:48.254179 2938 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:57:48.255357 2938 hybrid_clock.cc:648] HybridClock initialized: now 1786517868255337 us; error 38 us; skew 500 ppm
Aug 12 06:57:48 dist-test-slave-65mx krb5kdc[2923](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517868, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.52.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 06:57:48.258410 2938 init.cc:377] Logged in from keytab as kudu/127.19.52.126@KRBTEST.COM (short username kudu)
I20260812 06:57:48.259473 2938 webserver.cc:460] Webserver started at http://127.19.52.126:42859/ using document root <none> and password file <none>
I20260812 06:57:48.260087 2938 fs_manager.cc:359] Metadata directory not provided
I20260812 06:57:48.260198 2938 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:57:48.260449 2938 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 06:57:48.262735 2938 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517687384214-19665-0/minicluster-data/master-0/data/instance:
uuid: "cb9a783d342a46d4858bf6291d637d35"
format_stamp: "Formatted at 2026-08-12 06:57:48 on dist-test-slave-65mx"
server_key: "1165741c3419a9ea4d6292178791647f"
server_key_iv: "33a1a865a75e3f0cc43738e7bca493c4"
server_key_version: "encryptionkey@0"
I20260812 06:57:48.263384 2938 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517687384214-19665-0/minicluster-data/master-0/wal/instance:
uuid: "cb9a783d342a46d4858bf6291d637d35"
format_stamp: "Formatted at 2026-08-12 06:57:48 on dist-test-slave-65mx"
server_key: "1165741c3419a9ea4d6292178791647f"
server_key_iv: "33a1a865a75e3f0cc43738e7bca493c4"
server_key_version: "encryptionkey@0"
I20260812 06:57:48.267623 2938 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260812 06:57:48.270535 2957 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 06:57:48.271847 2938 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260812 06:57:48.272015 2938 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517687384214-19665-0/minicluster-data/master-0/data,/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517687384214-19665-0/minicluster-data/master-0/wal
uuid: "cb9a783d342a46d4858bf6291d637d35"
format_stamp: "Formatted at 2026-08-12 06:57:48 on dist-test-slave-65mx"
server_key: "1165741c3419a9ea4d6292178791647f"
server_key_iv: "33a1a865a75e3f0cc43738e7bca493c4"
server_key_version: "encryptionkey@0"
I20260812 06:57:48.272156 2938 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517687384214-19665-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517687384214-19665-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517687384214-19665-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:57:48.283735 2938 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 06:57:48.287181 2938 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 06:57:48.287423 2938 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:57:48.295490 2938 rpc_server.cc:307] RPC server started. Bound to: 127.19.52.126:42629
I20260812 06:57:48.295828 3019 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.52.126:42629 every 8 connection(s)
I20260812 06:57:48.296608 2938 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517687384214-19665-0/minicluster-data/master-0/data/info.pb
I20260812 06:57:48.299657 3020 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 06:57:48.303148 19665 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu as pid 2938
I20260812 06:57:48.303294 19665 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517687384214-19665-0/minicluster-data/master-0/wal/instance
I20260812 06:57:48.303633 19665 external_mini_cluster.cc:1428] Setting key 3b4f5e361e3383c06748b83dadbb4e55
I20260812 06:57:48.306376 3020 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cb9a783d342a46d4858bf6291d637d35: Bootstrap starting.
I20260812 06:57:48.308990 3020 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P cb9a783d342a46d4858bf6291d637d35: Neither blocks nor log segments found. Creating new log.
I20260812 06:57:48.309844 3020 log.cc:824] T 00000000000000000000000000000000 P cb9a783d342a46d4858bf6291d637d35: Log is configured to *not* fsync() on all Append() calls
I20260812 06:57:48.312148 3020 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cb9a783d342a46d4858bf6291d637d35: No bootstrap required, opened a new log
Aug 12 06:57:48 dist-test-slave-65mx krb5kdc[2923](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517868, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.19.52.126@KRBTEST.COM
I20260812 06:57:48.316931 3020 raft_consensus.cc:348] T 00000000000000000000000000000000 P cb9a783d342a46d4858bf6291d637d35 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cb9a783d342a46d4858bf6291d637d35" member_type: VOTER last_known_addr { host: "127.19.52.126" port: 42629 } }
I20260812 06:57:48.317204 3020 raft_consensus.cc:374] T 00000000000000000000000000000000 P cb9a783d342a46d4858bf6291d637d35 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 06:57:48.317282 3020 raft_consensus.cc:729] T 00000000000000000000000000000000 P cb9a783d342a46d4858bf6291d637d35 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cb9a783d342a46d4858bf6291d637d35, State: Initialized, Role: FOLLOWER
I20260812 06:57:48.317800 3020 consensus_queue.cc:260] T 00000000000000000000000000000000 P cb9a783d342a46d4858bf6291d637d35 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cb9a783d342a46d4858bf6291d637d35" member_type: VOTER last_known_addr { host: "127.19.52.126" port: 42629 } }
I20260812 06:57:48.317977 3020 raft_consensus.cc:388] T 00000000000000000000000000000000 P cb9a783d342a46d4858bf6291d637d35 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 06:57:48.318074 3020 raft_consensus.cc:482] T 00000000000000000000000000000000 P cb9a783d342a46d4858bf6291d637d35 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 06:57:48.318208 3020 raft_consensus.cc:3037] T 00000000000000000000000000000000 P cb9a783d342a46d4858bf6291d637d35 [term 0 FOLLOWER]: Advancing to term 1
I20260812 06:57:48.319561 3020 raft_consensus.cc:504] T 00000000000000000000000000000000 P cb9a783d342a46d4858bf6291d637d35 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cb9a783d342a46d4858bf6291d637d35" member_type: VOTER last_known_addr { host: "127.19.52.126" port: 42629 } }
I20260812 06:57:48.319939 3020 leader_election.cc:302] T 00000000000000000000000000000000 P cb9a783d342a46d4858bf6291d637d35 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: cb9a783d342a46d4858bf6291d637d35; no voters:
I20260812 06:57:48.320055 3023 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:57:48.305381 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44356 (local address 127.19.52.126:42629)
0812 06:57:48.305863 (+ 482us) server_negotiation.cc:206] Beginning negotiation
0812 06:57:48.305880 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0812 06:57:48.305917 (+ 37us) server_negotiation.cc:407] Connection header received
0812 06:57:48.306638 (+ 721us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:57:48.306656 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:57:48.306997 (+ 341us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:57:48.307351 (+ 354us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:57:48.308214 (+ 863us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:57:48.309350 (+ 1136us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:57:48.310274 (+ 924us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:57:48.310752 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:57:48.310938 (+ 186us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:57:48.314309 (+ 3371us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:57:48.314315 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:57:48.314331 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:57:48.314365 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:57:48.316407 (+ 2042us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:57:48.317014 (+ 607us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:57:48.317021 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:57:48.317026 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:57:48.317164 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:57:48.317542 (+ 378us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:57:48.317545 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:57:48.317552 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:57:48.318207 (+ 655us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:57:48.318455 (+ 248us) server_negotiation.cc:1035] Waiting for connection context
0812 06:57:48.318790 (+ 335us) server_negotiation.cc:299] Negotiation successful
0812 06:57:48.319147 (+ 357us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":326,"thread_start_us":171,"threads_started":1}
I20260812 06:57:48.320339 3020 leader_election.cc:288] T 00000000000000000000000000000000 P cb9a783d342a46d4858bf6291d637d35 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 06:57:48.320442 3025 raft_consensus.cc:2781] T 00000000000000000000000000000000 P cb9a783d342a46d4858bf6291d637d35 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 06:57:48.320675 3025 raft_consensus.cc:686] T 00000000000000000000000000000000 P cb9a783d342a46d4858bf6291d637d35 [term 1 LEADER]: Becoming Leader. State: Replica: cb9a783d342a46d4858bf6291d637d35, State: Running, Role: LEADER
I20260812 06:57:48.321069 3025 consensus_queue.cc:237] T 00000000000000000000000000000000 P cb9a783d342a46d4858bf6291d637d35 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cb9a783d342a46d4858bf6291d637d35" member_type: VOTER last_known_addr { host: "127.19.52.126" port: 42629 } }
I20260812 06:57:48.321768 3020 sys_catalog.cc:564] T 00000000000000000000000000000000 P cb9a783d342a46d4858bf6291d637d35 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 06:57:48.322352 3025 sys_catalog.cc:455] T 00000000000000000000000000000000 P cb9a783d342a46d4858bf6291d637d35 [sys.catalog]: SysCatalogTable state changed. Reason: New leader cb9a783d342a46d4858bf6291d637d35. Latest consensus state: current_term: 1 leader_uuid: "cb9a783d342a46d4858bf6291d637d35" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cb9a783d342a46d4858bf6291d637d35" member_type: VOTER last_known_addr { host: "127.19.52.126" port: 42629 } } }
I20260812 06:57:48.323328 3026 sys_catalog.cc:455] T 00000000000000000000000000000000 P cb9a783d342a46d4858bf6291d637d35 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "cb9a783d342a46d4858bf6291d637d35" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cb9a783d342a46d4858bf6291d637d35" member_type: VOTER last_known_addr { host: "127.19.52.126" port: 42629 } } }
I20260812 06:57:48.323477 3025 sys_catalog.cc:458] T 00000000000000000000000000000000 P cb9a783d342a46d4858bf6291d637d35 [sys.catalog]: This master's current role is: LEADER
I20260812 06:57:48.323559 3026 sys_catalog.cc:458] T 00000000000000000000000000000000 P cb9a783d342a46d4858bf6291d637d35 [sys.catalog]: This master's current role is: LEADER
I20260812 06:57:48.324384 3035 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 06:57:48.327553 3035 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 06:57:48.333957 3035 catalog_manager.cc:1347] Generated new cluster ID: e36be11f1aff48c080e5d4150585e637
I20260812 06:57:48.334051 3035 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 06:57:48.361138 3035 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 06:57:48.362502 3035 catalog_manager.cc:1504] Loading token signing keys...
I20260812 06:57:48.374671 3035 catalog_manager.cc:5868] T 00000000000000000000000000000000 P cb9a783d342a46d4858bf6291d637d35: Generated new TSK 0
I20260812 06:57:48.375674 3035 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.19.52.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.52.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.52.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517687384214-19665-0/krb5kdc/kudu_127.19.52.65.keytab.
Entry for principal kudu/127.19.52.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517687384214-19665-0/krb5kdc/kudu_127.19.52.65.keytab.
I20260812 06:57:48.417224 19665 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu
/tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517687384214-19665-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517687384214-19665-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517687384214-19665-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517687384214-19665-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.52.65:0
--local_ip_for_outbound_sockets=127.19.52.65
--webserver_interface=127.19.52.65
--webserver_port=0
--tserver_master_addrs=127.19.52.126:42629
--keytab_file=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517687384214-19665-0/krb5kdc/kudu_127.19.52.65.keytab
--principal=kudu/127.19.52.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.52.84:33591
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517687384214-19665-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517687384214-19665-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517687384214-19665-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 06:57:48.538096 3050 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:57:48.538443 3050 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:57:48.538527 3050 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 06:57:48.542863 3050 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 06:57:48.542976 3050 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 06:57:48.543071 3050 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.52.65
I20260812 06:57:48.548923 3050 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.52.84:33591
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517687384214-19665-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517687384214-19665-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517687384214-19665-0/krb5kdc/kudu_127.19.52.65.keytab
--principal=kudu/127.19.52.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.52.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517687384214-19665-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.19.52.65
--webserver_port=0
--tserver_master_addrs=127.19.52.126:42629
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3050
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.52.65
--log_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517687384214-19665-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 06:15:54 UTC on ccycloud.quasar-rnfzce.root.comops.site
build id 4581989
I20260812 06:57:48.550159 3050 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:57:48.551277 3050 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 06:57:48.561718 3057 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:57:48.561703 3059 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:57:48.561987 3050 server_base.cc:1034] running on GCE node
W20260812 06:57:48.561703 3056 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:57:48.562582 3050 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:57:48.563345 3050 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:57:48.564642 3050 hybrid_clock.cc:648] HybridClock initialized: now 1786517868564605 us; error 45 us; skew 500 ppm
Aug 12 06:57:48 dist-test-slave-65mx krb5kdc[2923](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517868, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.52.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 06:57:48.567687 3050 init.cc:377] Logged in from keytab as kudu/127.19.52.65@KRBTEST.COM (short username kudu)
I20260812 06:57:48.568729 3050 webserver.cc:460] Webserver started at http://127.19.52.65:35641/ using document root <none> and password file <none>
I20260812 06:57:48.569350 3050 fs_manager.cc:359] Metadata directory not provided
I20260812 06:57:48.569455 3050 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:57:48.569690 3050 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 06:57:48.571902 3050 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517687384214-19665-0/minicluster-data/ts-0/data/instance:
uuid: "53890c7692834a24bd8919a9cbae000a"
format_stamp: "Formatted at 2026-08-12 06:57:48 on dist-test-slave-65mx"
server_key: "7ff113d789f532f2b56580096c1e3507"
server_key_iv: "4b60a721ba9c2d1bca1c96839ab5cf2e"
server_key_version: "encryptionkey@0"
I20260812 06:57:48.572516 3050 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517687384214-19665-0/minicluster-data/ts-0/wal/instance:
uuid: "53890c7692834a24bd8919a9cbae000a"
format_stamp: "Formatted at 2026-08-12 06:57:48 on dist-test-slave-65mx"
server_key: "7ff113d789f532f2b56580096c1e3507"
server_key_iv: "4b60a721ba9c2d1bca1c96839ab5cf2e"
server_key_version: "encryptionkey@0"
I20260812 06:57:48.576776 3050 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260812 06:57:48.579489 3066 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 06:57:48.580856 3050 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 06:57:48.581008 3050 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517687384214-19665-0/minicluster-data/ts-0/data,/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517687384214-19665-0/minicluster-data/ts-0/wal
uuid: "53890c7692834a24bd8919a9cbae000a"
format_stamp: "Formatted at 2026-08-12 06:57:48 on dist-test-slave-65mx"
server_key: "7ff113d789f532f2b56580096c1e3507"
server_key_iv: "4b60a721ba9c2d1bca1c96839ab5cf2e"
server_key_version: "encryptionkey@0"
I20260812 06:57:48.581143 3050 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517687384214-19665-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517687384214-19665-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517687384214-19665-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:57:48.599774 3050 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 06:57:48.603746 3050 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 06:57:48.603996 3050 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:57:48.604692 3050 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 06:57:48.605600 3050 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 06:57:48.605682 3050 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 06:57:48.605754 3050 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 06:57:48.605809 3050 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 06:57:48.616909 3050 rpc_server.cc:307] RPC server started. Bound to: 127.19.52.65:43193
I20260812 06:57:48.616950 3203 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.52.65:43193 every 8 connection(s)
I20260812 06:57:48.617959 3050 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517687384214-19665-0/minicluster-data/ts-0/data/info.pb
I20260812 06:57:48.626680 19665 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu as pid 3050
I20260812 06:57:48.626844 19665 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517687384214-19665-0/minicluster-data/ts-0/wal/instance
I20260812 06:57:48.627228 19665 external_mini_cluster.cc:1428] Setting key 55db39fda3df18d89f4faa2346341f2d
Aug 12 06:57:48 dist-test-slave-65mx krb5kdc[2923](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517868, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.52.65@KRBTEST.COM for kudu/127.19.52.126@KRBTEST.COM
I20260812 06:57:48.634429 3023 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:57:48.619911 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.52.65:44119 (local address 127.19.52.126:42629)
0812 06:57:48.620108 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0812 06:57:48.620114 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 06:57:48.621046 (+ 932us) server_negotiation.cc:407] Connection header received
0812 06:57:48.622659 (+ 1613us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:57:48.622661 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:57:48.622713 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:57:48.622837 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:57:48.624271 (+ 1434us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:57:48.624911 (+ 640us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:57:48.626116 (+ 1205us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:57:48.626355 (+ 239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:57:48.626450 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:57:48.630199 (+ 3749us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:57:48.630203 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:57:48.630205 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:57:48.630231 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:57:48.631881 (+ 1650us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:57:48.632629 (+ 748us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:57:48.632634 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:57:48.632636 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:57:48.632733 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:57:48.633180 (+ 447us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:57:48.633183 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:57:48.633184 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:57:48.633381 (+ 197us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:57:48.633513 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0812 06:57:48.634089 (+ 576us) server_negotiation.cc:299] Negotiation successful
0812 06:57:48.634248 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260812 06:57:48.635311 3206 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:57:48.620301 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.126:42629 (local address 127.19.52.65:44119)
0812 06:57:48.620857 (+ 556us) negotiation.cc:107] Waiting for socket to connect
0812 06:57:48.620913 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0812 06:57:48.622379 (+ 1466us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:57:48.623036 (+ 657us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:57:48.623043 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:57:48.623588 (+ 545us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:57:48.624102 (+ 514us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:57:48.624124 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:57:48.625056 (+ 932us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:57:48.625059 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:57:48.625972 (+ 913us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:57:48.625981 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:57:48.626601 (+ 620us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:57:48.626607 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:57:48.626866 (+ 259us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:57:48.627660 (+ 794us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 06:57:48.627683 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:57:48.629962 (+ 2279us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 06:57:48.632032 (+ 2070us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:57:48.632038 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:57:48.632057 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:57:48.632366 (+ 309us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:57:48.632856 (+ 490us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:57:48.632859 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:57:48.632861 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:57:48.633030 (+ 169us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:57:48.633514 (+ 484us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 06:57:48.633524 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 06:57:48.633825 (+ 301us) client_negotiation.cc:769] Sending connection context
0812 06:57:48.634029 (+ 204us) client_negotiation.cc:240] Negotiation successful
0812 06:57:48.634399 (+ 370us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":360,"thread_start_us":132,"threads_started":1}
I20260812 06:57:48.636857 3204 heartbeater.cc:347] Connected to a master server at 127.19.52.126:42629
I20260812 06:57:48.637239 3204 heartbeater.cc:464] Registering TS with master...
I20260812 06:57:48.637936 3204 heartbeater.cc:511] Master 127.19.52.126:42629 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.19.52.66@KRBTEST.COM; defaulting to no policy
I20260812 06:57:48.639855 2979 ts_manager.cc:194] Registered new tserver with Master: 53890c7692834a24bd8919a9cbae000a (127.19.52.65:43193)
I20260812 06:57:48.641156 2979 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.52.65@KRBTEST.COM'} at 127.19.52.65:44119
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.52.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.52.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517687384214-19665-0/krb5kdc/kudu_127.19.52.66.keytab.
Entry for principal kudu/127.19.52.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517687384214-19665-0/krb5kdc/kudu_127.19.52.66.keytab.
I20260812 06:57:48.662056 19665 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu
/tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517687384214-19665-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517687384214-19665-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517687384214-19665-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517687384214-19665-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.52.66:0
--local_ip_for_outbound_sockets=127.19.52.66
--webserver_interface=127.19.52.66
--webserver_port=0
--tserver_master_addrs=127.19.52.126:42629
--keytab_file=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517687384214-19665-0/krb5kdc/kudu_127.19.52.66.keytab
--principal=kudu/127.19.52.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.52.84:33591
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517687384214-19665-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517687384214-19665-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517687384214-19665-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 06:57:48.781530 3210 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:57:48.781823 3210 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:57:48.781868 3210 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 06:57:48.786495 3210 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 06:57:48.786588 3210 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 06:57:48.786687 3210 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.52.66
I20260812 06:57:48.792658 3210 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.52.84:33591
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517687384214-19665-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517687384214-19665-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517687384214-19665-0/krb5kdc/kudu_127.19.52.66.keytab
--principal=kudu/127.19.52.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.52.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517687384214-19665-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.19.52.66
--webserver_port=0
--tserver_master_addrs=127.19.52.126:42629
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3210
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.52.66
--log_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517687384214-19665-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 06:15:54 UTC on ccycloud.quasar-rnfzce.root.comops.site
build id 4581989
I20260812 06:57:48.793848 3210 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:57:48.794950 3210 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 06:57:48.805686 3220 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:57:48.805671 3218 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:57:48.805671 3217 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:57:48.806169 3210 server_base.cc:1034] running on GCE node
I20260812 06:57:48.806648 3210 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:57:48.807442 3210 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:57:48.808651 3210 hybrid_clock.cc:648] HybridClock initialized: now 1786517868808633 us; error 36 us; skew 500 ppm
Aug 12 06:57:48 dist-test-slave-65mx krb5kdc[2923](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517868, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.52.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 06:57:48.811744 3210 init.cc:377] Logged in from keytab as kudu/127.19.52.66@KRBTEST.COM (short username kudu)
I20260812 06:57:48.812798 3210 webserver.cc:460] Webserver started at http://127.19.52.66:34873/ using document root <none> and password file <none>
I20260812 06:57:48.813390 3210 fs_manager.cc:359] Metadata directory not provided
I20260812 06:57:48.813503 3210 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:57:48.813786 3210 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 06:57:48.816046 3210 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517687384214-19665-0/minicluster-data/ts-1/data/instance:
uuid: "acc217e1270e4e56ac27f8bbe6ed20ba"
format_stamp: "Formatted at 2026-08-12 06:57:48 on dist-test-slave-65mx"
server_key: "1dd0239db6827471cbbe582197123db3"
server_key_iv: "2427b2eef86a471923318659774f55fb"
server_key_version: "encryptionkey@0"
I20260812 06:57:48.816749 3210 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517687384214-19665-0/minicluster-data/ts-1/wal/instance:
uuid: "acc217e1270e4e56ac27f8bbe6ed20ba"
format_stamp: "Formatted at 2026-08-12 06:57:48 on dist-test-slave-65mx"
server_key: "1dd0239db6827471cbbe582197123db3"
server_key_iv: "2427b2eef86a471923318659774f55fb"
server_key_version: "encryptionkey@0"
I20260812 06:57:48.821640 3210 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260812 06:57:48.824841 3228 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 06:57:48.826305 3210 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 06:57:48.826484 3210 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517687384214-19665-0/minicluster-data/ts-1/data,/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517687384214-19665-0/minicluster-data/ts-1/wal
uuid: "acc217e1270e4e56ac27f8bbe6ed20ba"
format_stamp: "Formatted at 2026-08-12 06:57:48 on dist-test-slave-65mx"
server_key: "1dd0239db6827471cbbe582197123db3"
server_key_iv: "2427b2eef86a471923318659774f55fb"
server_key_version: "encryptionkey@0"
I20260812 06:57:48.826650 3210 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517687384214-19665-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517687384214-19665-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517687384214-19665-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:57:48.839386 3210 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 06:57:48.842933 3210 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 06:57:48.843176 3210 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:57:48.843839 3210 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 06:57:48.844796 3210 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 06:57:48.844877 3210 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 06:57:48.844949 3210 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 06:57:48.845002 3210 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 06:57:48.854569 3210 rpc_server.cc:307] RPC server started. Bound to: 127.19.52.66:36889
I20260812 06:57:48.854601 3358 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.52.66:36889 every 8 connection(s)
I20260812 06:57:48.855521 3210 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517687384214-19665-0/minicluster-data/ts-1/data/info.pb
I20260812 06:57:48.860859 19665 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu as pid 3210
I20260812 06:57:48.860982 19665 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517687384214-19665-0/minicluster-data/ts-1/wal/instance
I20260812 06:57:48.861300 19665 external_mini_cluster.cc:1428] Setting key 37fa09b79ca85e5be194720bbd381799
Aug 12 06:57:48 dist-test-slave-65mx krb5kdc[2923](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517868, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.52.66@KRBTEST.COM for kudu/127.19.52.126@KRBTEST.COM
I20260812 06:57:48.872653 3023 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:57:48.857524 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.52.66:56653 (local address 127.19.52.126:42629)
0812 06:57:48.857707 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0812 06:57:48.857712 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 06:57:48.858599 (+ 887us) server_negotiation.cc:407] Connection header received
0812 06:57:48.860029 (+ 1430us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:57:48.860032 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:57:48.860086 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:57:48.860173 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:57:48.861667 (+ 1494us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:57:48.862309 (+ 642us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:57:48.863991 (+ 1682us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:57:48.864254 (+ 263us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:57:48.864355 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:57:48.867759 (+ 3404us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:57:48.867766 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:57:48.867772 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:57:48.867809 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:57:48.869957 (+ 2148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:57:48.870577 (+ 620us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:57:48.870582 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:57:48.870584 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:57:48.870703 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:57:48.871095 (+ 392us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:57:48.871102 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:57:48.871106 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:57:48.871421 (+ 315us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:57:48.871544 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0812 06:57:48.872208 (+ 664us) server_negotiation.cc:299] Negotiation successful
0812 06:57:48.872377 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260812 06:57:48.873490 3361 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:57:48.857934 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.126:42629 (local address 127.19.52.66:56653)
0812 06:57:48.858426 (+ 492us) negotiation.cc:107] Waiting for socket to connect
0812 06:57:48.858476 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0812 06:57:48.859803 (+ 1327us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:57:48.860417 (+ 614us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:57:48.860424 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:57:48.860835 (+ 411us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:57:48.861477 (+ 642us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:57:48.861494 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:57:48.862454 (+ 960us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:57:48.862457 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:57:48.863829 (+ 1372us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:57:48.863845 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:57:48.864443 (+ 598us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:57:48.864450 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:57:48.864751 (+ 301us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:57:48.865628 (+ 877us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 06:57:48.865658 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:57:48.867538 (+ 1880us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 06:57:48.870123 (+ 2585us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:57:48.870129 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:57:48.870145 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:57:48.870449 (+ 304us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:57:48.870820 (+ 371us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:57:48.870822 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:57:48.870824 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:57:48.870973 (+ 149us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:57:48.871578 (+ 605us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 06:57:48.871584 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 06:57:48.871872 (+ 288us) client_negotiation.cc:769] Sending connection context
0812 06:57:48.872097 (+ 225us) client_negotiation.cc:240] Negotiation successful
0812 06:57:48.872394 (+ 297us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":295,"thread_start_us":137,"threads_started":1}
WARNING: no policy specified for kudu/127.19.52.67@KRBTEST.COM; defaulting to no policy
I20260812 06:57:48.874951 3359 heartbeater.cc:347] Connected to a master server at 127.19.52.126:42629
I20260812 06:57:48.875223 3359 heartbeater.cc:464] Registering TS with master...
I20260812 06:57:48.875942 3359 heartbeater.cc:511] Master 127.19.52.126:42629 requested a full tablet report, sending...
I20260812 06:57:48.877660 2979 ts_manager.cc:194] Registered new tserver with Master: acc217e1270e4e56ac27f8bbe6ed20ba (127.19.52.66:36889)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.52.67@KRBTEST.COM" created.
I20260812 06:57:48.878582 2979 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.52.66@KRBTEST.COM'} at 127.19.52.66:56653
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.52.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517687384214-19665-0/krb5kdc/kudu_127.19.52.67.keytab.
Entry for principal kudu/127.19.52.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517687384214-19665-0/krb5kdc/kudu_127.19.52.67.keytab.
I20260812 06:57:48.896823 19665 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu
/tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517687384214-19665-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517687384214-19665-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517687384214-19665-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517687384214-19665-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.52.67:0
--local_ip_for_outbound_sockets=127.19.52.67
--webserver_interface=127.19.52.67
--webserver_port=0
--tserver_master_addrs=127.19.52.126:42629
--keytab_file=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517687384214-19665-0/krb5kdc/kudu_127.19.52.67.keytab
--principal=kudu/127.19.52.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.52.84:33591
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517687384214-19665-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517687384214-19665-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517687384214-19665-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 06:57:49.014577 3364 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:57:49.014859 3364 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:57:49.014904 3364 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 06:57:49.019277 3364 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 06:57:49.019353 3364 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 06:57:49.019416 3364 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.52.67
I20260812 06:57:49.025457 3364 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.52.84:33591
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517687384214-19665-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517687384214-19665-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517687384214-19665-0/krb5kdc/kudu_127.19.52.67.keytab
--principal=kudu/127.19.52.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.52.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517687384214-19665-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.19.52.67
--webserver_port=0
--tserver_master_addrs=127.19.52.126:42629
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3364
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.52.67
--log_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517687384214-19665-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 06:15:54 UTC on ccycloud.quasar-rnfzce.root.comops.site
build id 4581989
I20260812 06:57:49.026594 3364 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:57:49.027671 3364 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 06:57:49.038033 3369 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:57:49.038091 3364 server_base.cc:1034] running on GCE node
W20260812 06:57:49.038044 3370 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:57:49.038393 3373 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:57:49.038870 3364 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:57:49.039599 3364 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:57:49.040787 3364 hybrid_clock.cc:648] HybridClock initialized: now 1786517869040734 us; error 58 us; skew 500 ppm
Aug 12 06:57:49 dist-test-slave-65mx krb5kdc[2923](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517869, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.52.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 06:57:49.043869 3364 init.cc:377] Logged in from keytab as kudu/127.19.52.67@KRBTEST.COM (short username kudu)
I20260812 06:57:49.044997 3364 webserver.cc:460] Webserver started at http://127.19.52.67:42985/ using document root <none> and password file <none>
I20260812 06:57:49.045615 3364 fs_manager.cc:359] Metadata directory not provided
I20260812 06:57:49.045683 3364 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:57:49.045912 3364 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 06:57:49.048141 3364 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517687384214-19665-0/minicluster-data/ts-2/data/instance:
uuid: "c25a0ae5f88b4400a21eb4c9085cd2c3"
format_stamp: "Formatted at 2026-08-12 06:57:49 on dist-test-slave-65mx"
server_key: "87c1d3b0e65710d14c7e34951a9c243f"
server_key_iv: "1df5e0e999d968e6721cdfc759ff71bc"
server_key_version: "encryptionkey@0"
I20260812 06:57:49.048872 3364 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517687384214-19665-0/minicluster-data/ts-2/wal/instance:
uuid: "c25a0ae5f88b4400a21eb4c9085cd2c3"
format_stamp: "Formatted at 2026-08-12 06:57:49 on dist-test-slave-65mx"
server_key: "87c1d3b0e65710d14c7e34951a9c243f"
server_key_iv: "1df5e0e999d968e6721cdfc759ff71bc"
server_key_version: "encryptionkey@0"
I20260812 06:57:49.052946 3364 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260812 06:57:49.055609 3382 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 06:57:49.057081 3364 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 06:57:49.057245 3364 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517687384214-19665-0/minicluster-data/ts-2/data,/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517687384214-19665-0/minicluster-data/ts-2/wal
uuid: "c25a0ae5f88b4400a21eb4c9085cd2c3"
format_stamp: "Formatted at 2026-08-12 06:57:49 on dist-test-slave-65mx"
server_key: "87c1d3b0e65710d14c7e34951a9c243f"
server_key_iv: "1df5e0e999d968e6721cdfc759ff71bc"
server_key_version: "encryptionkey@0"
I20260812 06:57:49.057379 3364 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517687384214-19665-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517687384214-19665-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517687384214-19665-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:57:49.081017 3364 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 06:57:49.085074 3364 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 06:57:49.085320 3364 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:57:49.086125 3364 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 06:57:49.087112 3364 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 06:57:49.087199 3364 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 06:57:49.087275 3364 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 06:57:49.087332 3364 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 06:57:49.098222 3364 rpc_server.cc:307] RPC server started. Bound to: 127.19.52.67:46061
I20260812 06:57:49.098299 3509 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.52.67:46061 every 8 connection(s)
I20260812 06:57:49.099272 3364 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517687384214-19665-0/minicluster-data/ts-2/data/info.pb
I20260812 06:57:49.105564 19665 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu as pid 3364
I20260812 06:57:49.105773 19665 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517687384214-19665-0/minicluster-data/ts-2/wal/instance
I20260812 06:57:49.106101 19665 external_mini_cluster.cc:1428] Setting key adebf99acc7d3afb66541ebf30b60e15
Aug 12 06:57:49 dist-test-slave-65mx krb5kdc[2923](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517869, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.52.67@KRBTEST.COM for kudu/127.19.52.126@KRBTEST.COM
I20260812 06:57:49.116111 3023 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:57:49.101373 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.52.67:50953 (local address 127.19.52.126:42629)
0812 06:57:49.101547 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0812 06:57:49.101553 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 06:57:49.102486 (+ 933us) server_negotiation.cc:407] Connection header received
0812 06:57:49.103903 (+ 1417us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:57:49.103906 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:57:49.103961 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:57:49.104045 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:57:49.105685 (+ 1640us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:57:49.106568 (+ 883us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:57:49.108603 (+ 2035us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:57:49.108911 (+ 308us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:57:49.109051 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:57:49.111892 (+ 2841us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:57:49.111895 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:57:49.111898 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:57:49.111924 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:57:49.113777 (+ 1853us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:57:49.114416 (+ 639us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:57:49.114419 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:57:49.114420 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:57:49.114541 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:57:49.114959 (+ 418us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:57:49.114962 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:57:49.114964 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:57:49.115184 (+ 220us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:57:49.115275 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0812 06:57:49.115842 (+ 567us) server_negotiation.cc:299] Negotiation successful
0812 06:57:49.115969 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260812 06:57:49.117133 3512 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:57:49.101802 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.126:42629 (local address 127.19.52.67:50953)
0812 06:57:49.102314 (+ 512us) negotiation.cc:107] Waiting for socket to connect
0812 06:57:49.102359 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0812 06:57:49.103676 (+ 1317us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:57:49.104290 (+ 614us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:57:49.104299 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:57:49.104842 (+ 543us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:57:49.105467 (+ 625us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:57:49.105485 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:57:49.106748 (+ 1263us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:57:49.106754 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:57:49.108411 (+ 1657us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:57:49.108427 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:57:49.109090 (+ 663us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:57:49.109093 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:57:49.109269 (+ 176us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:57:49.110022 (+ 753us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 06:57:49.110042 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:57:49.111691 (+ 1649us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 06:57:49.113914 (+ 2223us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:57:49.113920 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:57:49.113972 (+ 52us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:57:49.114269 (+ 297us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:57:49.114667 (+ 398us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:57:49.114669 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:57:49.114671 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:57:49.114834 (+ 163us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:57:49.115306 (+ 472us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 06:57:49.115312 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 06:57:49.115598 (+ 286us) client_negotiation.cc:769] Sending connection context
0812 06:57:49.115772 (+ 174us) client_negotiation.cc:240] Negotiation successful
0812 06:57:49.116096 (+ 324us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":319,"thread_start_us":154,"threads_started":1}
I20260812 06:57:49.118589 3510 heartbeater.cc:347] Connected to a master server at 127.19.52.126:42629
I20260812 06:57:49.118988 3510 heartbeater.cc:464] Registering TS with master...
I20260812 06:57:49.119692 3510 heartbeater.cc:511] Master 127.19.52.126:42629 requested a full tablet report, sending...
I20260812 06:57:49.120884 2979 ts_manager.cc:194] Registered new tserver with Master: c25a0ae5f88b4400a21eb4c9085cd2c3 (127.19.52.67:46061)
I20260812 06:57:49.121542 2979 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.52.67@KRBTEST.COM'} at 127.19.52.67:50953
I20260812 06:57:49.133739 19665 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 06:57:49.144989 3023 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:57:49.136071 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44372 (local address 127.19.52.126:42629)
0812 06:57:49.136285 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0812 06:57:49.136290 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 06:57:49.136425 (+ 135us) server_negotiation.cc:407] Connection header received
0812 06:57:49.136674 (+ 249us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:57:49.136677 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:57:49.136727 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:57:49.136805 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:57:49.137652 (+ 847us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:57:49.138297 (+ 645us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:57:49.139228 (+ 931us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:57:49.139451 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:57:49.139568 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:57:49.141331 (+ 1763us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:57:49.141334 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:57:49.141337 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:57:49.141367 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:57:49.142999 (+ 1632us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:57:49.143550 (+ 551us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:57:49.143553 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:57:49.143554 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:57:49.143644 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:57:49.143973 (+ 329us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:57:49.143976 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:57:49.143978 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:57:49.144279 (+ 301us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:57:49.144412 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0812 06:57:49.144736 (+ 324us) server_negotiation.cc:299] Negotiation successful
0812 06:57:49.144850 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":99}
I20260812 06:57:49.149133 2979 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44372:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 06:57:49.152493 2979 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 06:57:49.169294 3526 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:57:49.163418 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.65:43193 (local address 127.0.0.1:51680)
0812 06:57:49.164089 (+ 671us) negotiation.cc:107] Waiting for socket to connect
0812 06:57:49.164103 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 06:57:49.164200 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:57:49.164923 (+ 723us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:57:49.164926 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:57:49.164944 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 06:57:49.165057 (+ 113us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:57:49.165066 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:57:49.166081 (+ 1015us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:57:49.166088 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:57:49.167520 (+ 1432us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:57:49.167530 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:57:49.168693 (+ 1163us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:57:49.168696 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:57:49.168832 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:57:49.168876 (+ 44us) client_negotiation.cc:769] Sending connection context
0812 06:57:49.168993 (+ 117us) client_negotiation.cc:240] Negotiation successful
0812 06:57:49.169120 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":613,"mutex_wait_us":157,"thread_start_us":101,"threads_started":1}
I20260812 06:57:49.169333 3527 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:57:49.163687 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51680 (local address 127.19.52.65:43193)
0812 06:57:49.164510 (+ 823us) server_negotiation.cc:206] Beginning negotiation
0812 06:57:49.164518 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 06:57:49.164544 (+ 26us) server_negotiation.cc:407] Connection header received
0812 06:57:49.164640 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:57:49.164652 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:57:49.164787 (+ 135us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:57:49.164946 (+ 159us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 06:57:49.165197 (+ 251us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:57:49.165950 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:57:49.167655 (+ 1705us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:57:49.168536 (+ 881us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:57:49.168697 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:57:49.168870 (+ 173us) server_negotiation.cc:1035] Waiting for connection context
0812 06:57:49.169052 (+ 182us) server_negotiation.cc:299] Negotiation successful
0812 06:57:49.169172 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":716,"thread_start_us":133,"threads_started":1}
I20260812 06:57:49.170004 3524 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:57:49.162821 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51732 (local address 127.19.52.66:36889)
0812 06:57:49.163524 (+ 703us) server_negotiation.cc:206] Beginning negotiation
0812 06:57:49.163537 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0812 06:57:49.163853 (+ 316us) server_negotiation.cc:407] Connection header received
0812 06:57:49.163930 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:57:49.163935 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:57:49.164126 (+ 191us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:57:49.164261 (+ 135us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 06:57:49.165017 (+ 756us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:57:49.166253 (+ 1236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:57:49.167806 (+ 1553us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:57:49.168656 (+ 850us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:57:49.168746 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:57:49.168870 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0812 06:57:49.169371 (+ 501us) server_negotiation.cc:299] Negotiation successful
0812 06:57:49.169812 (+ 441us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":592,"thread_start_us":123,"threads_started":1}
I20260812 06:57:49.170225 3522 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:57:49.162635 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.67:46061 (local address 127.0.0.1:44694)
0812 06:57:49.163491 (+ 856us) negotiation.cc:107] Waiting for socket to connect
0812 06:57:49.163551 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0812 06:57:49.163725 (+ 174us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:57:49.164452 (+ 727us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:57:49.164458 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:57:49.164499 (+ 41us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 06:57:49.164865 (+ 366us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:57:49.164874 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:57:49.167786 (+ 2912us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:57:49.167788 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:57:49.168476 (+ 688us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:57:49.168482 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:57:49.169826 (+ 1344us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:57:49.169830 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:57:49.169927 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:57:49.169950 (+ 23us) client_negotiation.cc:769] Sending connection context
0812 06:57:49.170019 (+ 69us) client_negotiation.cc:240] Negotiation successful
0812 06:57:49.170085 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":788,"mutex_wait_us":226,"spinlock_wait_cycles":4224,"thread_start_us":106,"threads_started":1}
I20260812 06:57:49.170665 3525 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:57:49.162875 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44694 (local address 127.19.52.67:46061)
0812 06:57:49.163349 (+ 474us) server_negotiation.cc:206] Beginning negotiation
0812 06:57:49.163362 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0812 06:57:49.163661 (+ 299us) server_negotiation.cc:407] Connection header received
0812 06:57:49.163898 (+ 237us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:57:49.163904 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:57:49.164115 (+ 211us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:57:49.164261 (+ 146us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 06:57:49.165017 (+ 756us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:57:49.166253 (+ 1236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:57:49.168636 (+ 2383us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:57:49.169486 (+ 850us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:57:49.169598 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:57:49.169759 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0812 06:57:49.170420 (+ 661us) server_negotiation.cc:299] Negotiation successful
0812 06:57:49.170522 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":366,"thread_start_us":157,"threads_started":1}
I20260812 06:57:49.170979 3523 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:57:49.162632 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.66:36889 (local address 127.0.0.1:51732)
0812 06:57:49.163491 (+ 859us) negotiation.cc:107] Waiting for socket to connect
0812 06:57:49.163551 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0812 06:57:49.163709 (+ 158us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:57:49.164457 (+ 748us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:57:49.164459 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:57:49.164495 (+ 36us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 06:57:49.164865 (+ 370us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:57:49.164874 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:57:49.166589 (+ 1715us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:57:49.166592 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:57:49.167691 (+ 1099us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:57:49.167698 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:57:49.168943 (+ 1245us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:57:49.168946 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:57:49.169077 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:57:49.169101 (+ 24us) client_negotiation.cc:769] Sending connection context
0812 06:57:49.169183 (+ 82us) client_negotiation.cc:240] Negotiation successful
0812 06:57:49.169255 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":679,"mutex_wait_us":90,"spinlock_wait_cycles":9216,"thread_start_us":114,"threads_started":1}
I20260812 06:57:49.172829 3131 tablet_service.cc:1467] Processing CreateTablet for tablet 8d66008fbc174517b82ef15f30b943e8 (DEFAULT_TABLE table=test-table [id=fd66bc65982f4c809fb145056825e5f7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 06:57:49.173624 3437 tablet_service.cc:1467] Processing CreateTablet for tablet 8d66008fbc174517b82ef15f30b943e8 (DEFAULT_TABLE table=test-table [id=fd66bc65982f4c809fb145056825e5f7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 06:57:49.174086 3131 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8d66008fbc174517b82ef15f30b943e8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 06:57:49.174695 3437 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8d66008fbc174517b82ef15f30b943e8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 06:57:49.174327 3283 tablet_service.cc:1467] Processing CreateTablet for tablet 8d66008fbc174517b82ef15f30b943e8 (DEFAULT_TABLE table=test-table [id=fd66bc65982f4c809fb145056825e5f7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 06:57:49.175428 3283 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8d66008fbc174517b82ef15f30b943e8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 06:57:49.181810 3528 tablet_bootstrap.cc:492] T 8d66008fbc174517b82ef15f30b943e8 P acc217e1270e4e56ac27f8bbe6ed20ba: Bootstrap starting.
I20260812 06:57:49.182883 3530 tablet_bootstrap.cc:492] T 8d66008fbc174517b82ef15f30b943e8 P c25a0ae5f88b4400a21eb4c9085cd2c3: Bootstrap starting.
I20260812 06:57:49.183341 3529 tablet_bootstrap.cc:492] T 8d66008fbc174517b82ef15f30b943e8 P 53890c7692834a24bd8919a9cbae000a: Bootstrap starting.
I20260812 06:57:49.185571 3530 tablet_bootstrap.cc:654] T 8d66008fbc174517b82ef15f30b943e8 P c25a0ae5f88b4400a21eb4c9085cd2c3: Neither blocks nor log segments found. Creating new log.
I20260812 06:57:49.185726 3529 tablet_bootstrap.cc:654] T 8d66008fbc174517b82ef15f30b943e8 P 53890c7692834a24bd8919a9cbae000a: Neither blocks nor log segments found. Creating new log.
I20260812 06:57:49.186735 3530 log.cc:824] T 8d66008fbc174517b82ef15f30b943e8 P c25a0ae5f88b4400a21eb4c9085cd2c3: Log is configured to *not* fsync() on all Append() calls
I20260812 06:57:49.186709 3529 log.cc:824] T 8d66008fbc174517b82ef15f30b943e8 P 53890c7692834a24bd8919a9cbae000a: Log is configured to *not* fsync() on all Append() calls
I20260812 06:57:49.187325 3528 tablet_bootstrap.cc:654] T 8d66008fbc174517b82ef15f30b943e8 P acc217e1270e4e56ac27f8bbe6ed20ba: Neither blocks nor log segments found. Creating new log.
I20260812 06:57:49.188248 3528 log.cc:824] T 8d66008fbc174517b82ef15f30b943e8 P acc217e1270e4e56ac27f8bbe6ed20ba: Log is configured to *not* fsync() on all Append() calls
I20260812 06:57:49.189221 3530 tablet_bootstrap.cc:492] T 8d66008fbc174517b82ef15f30b943e8 P c25a0ae5f88b4400a21eb4c9085cd2c3: No bootstrap required, opened a new log
I20260812 06:57:49.189450 3530 ts_tablet_manager.cc:1397] T 8d66008fbc174517b82ef15f30b943e8 P c25a0ae5f88b4400a21eb4c9085cd2c3: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260812 06:57:49.189774 3529 tablet_bootstrap.cc:492] T 8d66008fbc174517b82ef15f30b943e8 P 53890c7692834a24bd8919a9cbae000a: No bootstrap required, opened a new log
I20260812 06:57:49.189980 3529 ts_tablet_manager.cc:1397] T 8d66008fbc174517b82ef15f30b943e8 P 53890c7692834a24bd8919a9cbae000a: Time spent bootstrapping tablet: real 0.007s user 0.004s sys 0.000s
I20260812 06:57:49.190415 3528 tablet_bootstrap.cc:492] T 8d66008fbc174517b82ef15f30b943e8 P acc217e1270e4e56ac27f8bbe6ed20ba: No bootstrap required, opened a new log
I20260812 06:57:49.190604 3528 ts_tablet_manager.cc:1397] T 8d66008fbc174517b82ef15f30b943e8 P acc217e1270e4e56ac27f8bbe6ed20ba: Time spent bootstrapping tablet: real 0.009s user 0.002s sys 0.003s
I20260812 06:57:49.193653 3529 raft_consensus.cc:348] T 8d66008fbc174517b82ef15f30b943e8 P 53890c7692834a24bd8919a9cbae000a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acc217e1270e4e56ac27f8bbe6ed20ba" member_type: VOTER last_known_addr { host: "127.19.52.66" port: 36889 } } peers { permanent_uuid: "c25a0ae5f88b4400a21eb4c9085cd2c3" member_type: VOTER last_known_addr { host: "127.19.52.67" port: 46061 } } peers { permanent_uuid: "53890c7692834a24bd8919a9cbae000a" member_type: VOTER last_known_addr { host: "127.19.52.65" port: 43193 } }
I20260812 06:57:49.193636 3530 raft_consensus.cc:348] T 8d66008fbc174517b82ef15f30b943e8 P c25a0ae5f88b4400a21eb4c9085cd2c3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acc217e1270e4e56ac27f8bbe6ed20ba" member_type: VOTER last_known_addr { host: "127.19.52.66" port: 36889 } } peers { permanent_uuid: "c25a0ae5f88b4400a21eb4c9085cd2c3" member_type: VOTER last_known_addr { host: "127.19.52.67" port: 46061 } } peers { permanent_uuid: "53890c7692834a24bd8919a9cbae000a" member_type: VOTER last_known_addr { host: "127.19.52.65" port: 43193 } }
I20260812 06:57:49.193929 3530 raft_consensus.cc:374] T 8d66008fbc174517b82ef15f30b943e8 P c25a0ae5f88b4400a21eb4c9085cd2c3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 06:57:49.193930 3529 raft_consensus.cc:374] T 8d66008fbc174517b82ef15f30b943e8 P 53890c7692834a24bd8919a9cbae000a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 06:57:49.193991 3529 raft_consensus.cc:729] T 8d66008fbc174517b82ef15f30b943e8 P 53890c7692834a24bd8919a9cbae000a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 53890c7692834a24bd8919a9cbae000a, State: Initialized, Role: FOLLOWER
I20260812 06:57:49.193991 3530 raft_consensus.cc:729] T 8d66008fbc174517b82ef15f30b943e8 P c25a0ae5f88b4400a21eb4c9085cd2c3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c25a0ae5f88b4400a21eb4c9085cd2c3, State: Initialized, Role: FOLLOWER
I20260812 06:57:49.194468 3530 consensus_queue.cc:260] T 8d66008fbc174517b82ef15f30b943e8 P c25a0ae5f88b4400a21eb4c9085cd2c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acc217e1270e4e56ac27f8bbe6ed20ba" member_type: VOTER last_known_addr { host: "127.19.52.66" port: 36889 } } peers { permanent_uuid: "c25a0ae5f88b4400a21eb4c9085cd2c3" member_type: VOTER last_known_addr { host: "127.19.52.67" port: 46061 } } peers { permanent_uuid: "53890c7692834a24bd8919a9cbae000a" member_type: VOTER last_known_addr { host: "127.19.52.65" port: 43193 } }
I20260812 06:57:49.194700 3529 consensus_queue.cc:260] T 8d66008fbc174517b82ef15f30b943e8 P 53890c7692834a24bd8919a9cbae000a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acc217e1270e4e56ac27f8bbe6ed20ba" member_type: VOTER last_known_addr { host: "127.19.52.66" port: 36889 } } peers { permanent_uuid: "c25a0ae5f88b4400a21eb4c9085cd2c3" member_type: VOTER last_known_addr { host: "127.19.52.67" port: 46061 } } peers { permanent_uuid: "53890c7692834a24bd8919a9cbae000a" member_type: VOTER last_known_addr { host: "127.19.52.65" port: 43193 } }
I20260812 06:57:49.195377 3510 heartbeater.cc:503] Master 127.19.52.126:42629 was elected leader, sending a full tablet report...
I20260812 06:57:49.195775 3529 ts_tablet_manager.cc:1428] T 8d66008fbc174517b82ef15f30b943e8 P 53890c7692834a24bd8919a9cbae000a: Time spent starting tablet: real 0.006s user 0.003s sys 0.003s
I20260812 06:57:49.195874 3528 raft_consensus.cc:348] T 8d66008fbc174517b82ef15f30b943e8 P acc217e1270e4e56ac27f8bbe6ed20ba [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acc217e1270e4e56ac27f8bbe6ed20ba" member_type: VOTER last_known_addr { host: "127.19.52.66" port: 36889 } } peers { permanent_uuid: "c25a0ae5f88b4400a21eb4c9085cd2c3" member_type: VOTER last_known_addr { host: "127.19.52.67" port: 46061 } } peers { permanent_uuid: "53890c7692834a24bd8919a9cbae000a" member_type: VOTER last_known_addr { host: "127.19.52.65" port: 43193 } }
I20260812 06:57:49.196148 3528 raft_consensus.cc:374] T 8d66008fbc174517b82ef15f30b943e8 P acc217e1270e4e56ac27f8bbe6ed20ba [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 06:57:49.196151 3204 heartbeater.cc:503] Master 127.19.52.126:42629 was elected leader, sending a full tablet report...
I20260812 06:57:49.196210 3528 raft_consensus.cc:729] T 8d66008fbc174517b82ef15f30b943e8 P acc217e1270e4e56ac27f8bbe6ed20ba [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: acc217e1270e4e56ac27f8bbe6ed20ba, State: Initialized, Role: FOLLOWER
I20260812 06:57:49.196728 3530 ts_tablet_manager.cc:1428] T 8d66008fbc174517b82ef15f30b943e8 P c25a0ae5f88b4400a21eb4c9085cd2c3: Time spent starting tablet: real 0.007s user 0.007s sys 0.000s
I20260812 06:57:49.196686 3528 consensus_queue.cc:260] T 8d66008fbc174517b82ef15f30b943e8 P acc217e1270e4e56ac27f8bbe6ed20ba [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acc217e1270e4e56ac27f8bbe6ed20ba" member_type: VOTER last_known_addr { host: "127.19.52.66" port: 36889 } } peers { permanent_uuid: "c25a0ae5f88b4400a21eb4c9085cd2c3" member_type: VOTER last_known_addr { host: "127.19.52.67" port: 46061 } } peers { permanent_uuid: "53890c7692834a24bd8919a9cbae000a" member_type: VOTER last_known_addr { host: "127.19.52.65" port: 43193 } }
I20260812 06:57:49.197458 3359 heartbeater.cc:503] Master 127.19.52.126:42629 was elected leader, sending a full tablet report...
I20260812 06:57:49.197669 3528 ts_tablet_manager.cc:1428] T 8d66008fbc174517b82ef15f30b943e8 P acc217e1270e4e56ac27f8bbe6ed20ba: Time spent starting tablet: real 0.007s user 0.003s sys 0.003s
W20260812 06:57:49.350404 3511 tablet.cc:2369] T 8d66008fbc174517b82ef15f30b943e8 P c25a0ae5f88b4400a21eb4c9085cd2c3: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 06:57:49.356870 3360 tablet.cc:2369] T 8d66008fbc174517b82ef15f30b943e8 P acc217e1270e4e56ac27f8bbe6ed20ba: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 06:57:49.369586 3205 tablet.cc:2369] T 8d66008fbc174517b82ef15f30b943e8 P 53890c7692834a24bd8919a9cbae000a: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 06:57:49.433883 3536 raft_consensus.cc:482] T 8d66008fbc174517b82ef15f30b943e8 P acc217e1270e4e56ac27f8bbe6ed20ba [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 06:57:49.434141 3536 raft_consensus.cc:504] T 8d66008fbc174517b82ef15f30b943e8 P acc217e1270e4e56ac27f8bbe6ed20ba [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acc217e1270e4e56ac27f8bbe6ed20ba" member_type: VOTER last_known_addr { host: "127.19.52.66" port: 36889 } } peers { permanent_uuid: "c25a0ae5f88b4400a21eb4c9085cd2c3" member_type: VOTER last_known_addr { host: "127.19.52.67" port: 46061 } } peers { permanent_uuid: "53890c7692834a24bd8919a9cbae000a" member_type: VOTER last_known_addr { host: "127.19.52.65" port: 43193 } }
I20260812 06:57:49.435561 3536 leader_election.cc:288] T 8d66008fbc174517b82ef15f30b943e8 P acc217e1270e4e56ac27f8bbe6ed20ba [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c25a0ae5f88b4400a21eb4c9085cd2c3 (127.19.52.67:46061), 53890c7692834a24bd8919a9cbae000a (127.19.52.65:43193)
I20260812 06:57:49.439821 3525 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:57:49.435905 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.52.66:60757 (local address 127.19.52.67:46061)
0812 06:57:49.436096 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0812 06:57:49.436101 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 06:57:49.436166 (+ 65us) server_negotiation.cc:407] Connection header received
0812 06:57:49.436368 (+ 202us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:57:49.436372 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:57:49.436436 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:57:49.436514 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 06:57:49.436936 (+ 422us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:57:49.437510 (+ 574us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:57:49.438738 (+ 1228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:57:49.439178 (+ 440us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:57:49.439326 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:57:49.439385 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0812 06:57:49.439618 (+ 233us) server_negotiation.cc:299] Negotiation successful
0812 06:57:49.439695 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260812 06:57:49.439838 3537 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:57:49.435770 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.67:46061 (local address 127.19.52.66:60757)
0812 06:57:49.436078 (+ 308us) negotiation.cc:107] Waiting for socket to connect
0812 06:57:49.436096 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0812 06:57:49.436227 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:57:49.436598 (+ 371us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:57:49.436602 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:57:49.436632 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 06:57:49.436778 (+ 146us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:57:49.436786 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:57:49.437655 (+ 869us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:57:49.437658 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:57:49.438542 (+ 884us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:57:49.438552 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:57:49.439326 (+ 774us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:57:49.439330 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:57:49.439466 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:57:49.439487 (+ 21us) client_negotiation.cc:769] Sending connection context
0812 06:57:49.439588 (+ 101us) client_negotiation.cc:240] Negotiation successful
0812 06:57:49.439676 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":221,"mutex_wait_us":3,"thread_start_us":125,"threads_started":1}
I20260812 06:57:49.439945 3527 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:57:49.436353 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.52.66:35799 (local address 127.19.52.65:43193)
0812 06:57:49.436532 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0812 06:57:49.436537 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 06:57:49.436552 (+ 15us) server_negotiation.cc:407] Connection header received
0812 06:57:49.436628 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:57:49.436631 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:57:49.436676 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:57:49.436760 (+ 84us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 06:57:49.437103 (+ 343us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:57:49.437691 (+ 588us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:57:49.438814 (+ 1123us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:57:49.439277 (+ 463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:57:49.439425 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:57:49.439483 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0812 06:57:49.439736 (+ 253us) server_negotiation.cc:299] Negotiation successful
0812 06:57:49.439818 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260812 06:57:49.440013 3538 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:57:49.435896 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.65:43193 (local address 127.19.52.66:35799)
0812 06:57:49.436206 (+ 310us) negotiation.cc:107] Waiting for socket to connect
0812 06:57:49.436218 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 06:57:49.436301 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:57:49.436784 (+ 483us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:57:49.436787 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:57:49.436804 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 06:57:49.436977 (+ 173us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:57:49.436985 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:57:49.437809 (+ 824us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:57:49.437812 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:57:49.438587 (+ 775us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:57:49.438597 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:57:49.439425 (+ 828us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:57:49.439429 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:57:49.439600 (+ 171us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:57:49.439620 (+ 20us) client_negotiation.cc:769] Sending connection context
0812 06:57:49.439717 (+ 97us) client_negotiation.cc:240] Negotiation successful
0812 06:57:49.439791 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":255,"mutex_wait_us":49,"thread_start_us":72,"threads_started":1}
I20260812 06:57:49.440691 3459 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8d66008fbc174517b82ef15f30b943e8" candidate_uuid: "acc217e1270e4e56ac27f8bbe6ed20ba" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c25a0ae5f88b4400a21eb4c9085cd2c3" is_pre_election: true
I20260812 06:57:49.440868 3153 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8d66008fbc174517b82ef15f30b943e8" candidate_uuid: "acc217e1270e4e56ac27f8bbe6ed20ba" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "53890c7692834a24bd8919a9cbae000a" is_pre_election: true
I20260812 06:57:49.441090 3459 raft_consensus.cc:2454] T 8d66008fbc174517b82ef15f30b943e8 P c25a0ae5f88b4400a21eb4c9085cd2c3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acc217e1270e4e56ac27f8bbe6ed20ba in term 0.
I20260812 06:57:49.441197 3153 raft_consensus.cc:2454] T 8d66008fbc174517b82ef15f30b943e8 P 53890c7692834a24bd8919a9cbae000a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acc217e1270e4e56ac27f8bbe6ed20ba in term 0.
I20260812 06:57:49.441648 3231 leader_election.cc:302] T 8d66008fbc174517b82ef15f30b943e8 P acc217e1270e4e56ac27f8bbe6ed20ba [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 53890c7692834a24bd8919a9cbae000a, acc217e1270e4e56ac27f8bbe6ed20ba, c25a0ae5f88b4400a21eb4c9085cd2c3; no voters:
I20260812 06:57:49.442003 3536 raft_consensus.cc:2781] T 8d66008fbc174517b82ef15f30b943e8 P acc217e1270e4e56ac27f8bbe6ed20ba [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 06:57:49.442126 3536 raft_consensus.cc:482] T 8d66008fbc174517b82ef15f30b943e8 P acc217e1270e4e56ac27f8bbe6ed20ba [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 06:57:49.442191 3536 raft_consensus.cc:3037] T 8d66008fbc174517b82ef15f30b943e8 P acc217e1270e4e56ac27f8bbe6ed20ba [term 0 FOLLOWER]: Advancing to term 1
I20260812 06:57:49.443352 3536 raft_consensus.cc:504] T 8d66008fbc174517b82ef15f30b943e8 P acc217e1270e4e56ac27f8bbe6ed20ba [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acc217e1270e4e56ac27f8bbe6ed20ba" member_type: VOTER last_known_addr { host: "127.19.52.66" port: 36889 } } peers { permanent_uuid: "c25a0ae5f88b4400a21eb4c9085cd2c3" member_type: VOTER last_known_addr { host: "127.19.52.67" port: 46061 } } peers { permanent_uuid: "53890c7692834a24bd8919a9cbae000a" member_type: VOTER last_known_addr { host: "127.19.52.65" port: 43193 } }
I20260812 06:57:49.443745 3536 leader_election.cc:288] T 8d66008fbc174517b82ef15f30b943e8 P acc217e1270e4e56ac27f8bbe6ed20ba [CANDIDATE]: Term 1 election: Requested vote from peers c25a0ae5f88b4400a21eb4c9085cd2c3 (127.19.52.67:46061), 53890c7692834a24bd8919a9cbae000a (127.19.52.65:43193)
I20260812 06:57:49.444192 3153 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8d66008fbc174517b82ef15f30b943e8" candidate_uuid: "acc217e1270e4e56ac27f8bbe6ed20ba" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "53890c7692834a24bd8919a9cbae000a"
I20260812 06:57:49.444216 3459 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8d66008fbc174517b82ef15f30b943e8" candidate_uuid: "acc217e1270e4e56ac27f8bbe6ed20ba" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c25a0ae5f88b4400a21eb4c9085cd2c3"
I20260812 06:57:49.444339 3459 raft_consensus.cc:3037] T 8d66008fbc174517b82ef15f30b943e8 P c25a0ae5f88b4400a21eb4c9085cd2c3 [term 0 FOLLOWER]: Advancing to term 1
I20260812 06:57:49.444347 3153 raft_consensus.cc:3037] T 8d66008fbc174517b82ef15f30b943e8 P 53890c7692834a24bd8919a9cbae000a [term 0 FOLLOWER]: Advancing to term 1
I20260812 06:57:49.445432 3459 raft_consensus.cc:2454] T 8d66008fbc174517b82ef15f30b943e8 P c25a0ae5f88b4400a21eb4c9085cd2c3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acc217e1270e4e56ac27f8bbe6ed20ba in term 1.
I20260812 06:57:49.445449 3153 raft_consensus.cc:2454] T 8d66008fbc174517b82ef15f30b943e8 P 53890c7692834a24bd8919a9cbae000a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acc217e1270e4e56ac27f8bbe6ed20ba in term 1.
I20260812 06:57:49.445745 3231 leader_election.cc:302] T 8d66008fbc174517b82ef15f30b943e8 P acc217e1270e4e56ac27f8bbe6ed20ba [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: acc217e1270e4e56ac27f8bbe6ed20ba, c25a0ae5f88b4400a21eb4c9085cd2c3; no voters:
I20260812 06:57:49.445935 3536 raft_consensus.cc:2781] T 8d66008fbc174517b82ef15f30b943e8 P acc217e1270e4e56ac27f8bbe6ed20ba [term 1 FOLLOWER]: Leader election won for term 1
I20260812 06:57:49.446175 3536 raft_consensus.cc:686] T 8d66008fbc174517b82ef15f30b943e8 P acc217e1270e4e56ac27f8bbe6ed20ba [term 1 LEADER]: Becoming Leader. State: Replica: acc217e1270e4e56ac27f8bbe6ed20ba, State: Running, Role: LEADER
I20260812 06:57:49.446540 3536 consensus_queue.cc:237] T 8d66008fbc174517b82ef15f30b943e8 P acc217e1270e4e56ac27f8bbe6ed20ba [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acc217e1270e4e56ac27f8bbe6ed20ba" member_type: VOTER last_known_addr { host: "127.19.52.66" port: 36889 } } peers { permanent_uuid: "c25a0ae5f88b4400a21eb4c9085cd2c3" member_type: VOTER last_known_addr { host: "127.19.52.67" port: 46061 } } peers { permanent_uuid: "53890c7692834a24bd8919a9cbae000a" member_type: VOTER last_known_addr { host: "127.19.52.65" port: 43193 } }
I20260812 06:57:49.450311 2978 catalog_manager.cc:5495] T 8d66008fbc174517b82ef15f30b943e8 P acc217e1270e4e56ac27f8bbe6ed20ba reported cstate change: term changed from 0 to 1, leader changed from <none> to acc217e1270e4e56ac27f8bbe6ed20ba (127.19.52.66). New cstate: current_term: 1 leader_uuid: "acc217e1270e4e56ac27f8bbe6ed20ba" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acc217e1270e4e56ac27f8bbe6ed20ba" member_type: VOTER last_known_addr { host: "127.19.52.66" port: 36889 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c25a0ae5f88b4400a21eb4c9085cd2c3" member_type: VOTER last_known_addr { host: "127.19.52.67" port: 46061 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "53890c7692834a24bd8919a9cbae000a" member_type: VOTER last_known_addr { host: "127.19.52.65" port: 43193 } health_report { overall_health: UNKNOWN } } }
I20260812 06:57:49.465013 3524 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:57:49.461341 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51738 (local address 127.19.52.66:36889)
0812 06:57:49.461475 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0812 06:57:49.461478 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 06:57:49.461493 (+ 15us) server_negotiation.cc:407] Connection header received
0812 06:57:49.461622 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:57:49.461654 (+ 32us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:57:49.461727 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:57:49.461815 (+ 88us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 06:57:49.462071 (+ 256us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:57:49.462576 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:57:49.463452 (+ 876us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:57:49.463675 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:57:49.463756 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:57:49.464056 (+ 300us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 06:57:49.464503 (+ 447us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 06:57:49.464697 (+ 194us) server_negotiation.cc:1035] Waiting for connection context
0812 06:57:49.464770 (+ 73us) server_negotiation.cc:299] Negotiation successful
0812 06:57:49.464827 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260812 06:57:49.471446 3459 raft_consensus.cc:1261] T 8d66008fbc174517b82ef15f30b943e8 P c25a0ae5f88b4400a21eb4c9085cd2c3 [term 1 FOLLOWER]: Refusing update from remote peer acc217e1270e4e56ac27f8bbe6ed20ba: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 06:57:49.471517 3153 raft_consensus.cc:1261] T 8d66008fbc174517b82ef15f30b943e8 P 53890c7692834a24bd8919a9cbae000a [term 1 FOLLOWER]: Refusing update from remote peer acc217e1270e4e56ac27f8bbe6ed20ba: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 06:57:49.472304 3536 consensus_queue.cc:1035] T 8d66008fbc174517b82ef15f30b943e8 P acc217e1270e4e56ac27f8bbe6ed20ba [LEADER]: Connected to new peer: Peer: permanent_uuid: "c25a0ae5f88b4400a21eb4c9085cd2c3" member_type: VOTER last_known_addr { host: "127.19.52.67" port: 46061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 06:57:49.472604 3539 consensus_queue.cc:1035] T 8d66008fbc174517b82ef15f30b943e8 P acc217e1270e4e56ac27f8bbe6ed20ba [LEADER]: Connected to new peer: Peer: permanent_uuid: "53890c7692834a24bd8919a9cbae000a" member_type: VOTER last_known_addr { host: "127.19.52.65" port: 43193 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 06:57:49.479924 3542 mvcc.cc:205] Tried to move back new op lower bound from 7317577193346252800 to 7317577193255927808. Current Snapshot: MvccSnapshot[applied={T|T < 7317577193346252800}]
I20260812 06:57:49.481082 3543 mvcc.cc:205] Tried to move back new op lower bound from 7317577193346252800 to 7317577193255927808. Current Snapshot: MvccSnapshot[applied={T|T < 7317577193346252800}]
I20260812 06:57:49.490636 2976 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44372:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 06:57:49.490860 2976 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44372:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 06:57:49.493876 2976 catalog_manager.cc:5782] T 00000000000000000000000000000000 P cb9a783d342a46d4858bf6291d637d35: Sending DeleteTablet for 3 replicas of tablet 8d66008fbc174517b82ef15f30b943e8
I20260812 06:57:49.494988 3437 tablet_service.cc:1514] Processing DeleteTablet for tablet 8d66008fbc174517b82ef15f30b943e8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 06:57:49 UTC) from {username='kudu', principal='kudu/127.19.52.126@KRBTEST.COM'} at 127.0.0.1:44694
I20260812 06:57:49.495113 19665 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu with pid 3050
I20260812 06:57:49.495595 3283 tablet_service.cc:1514] Processing DeleteTablet for tablet 8d66008fbc174517b82ef15f30b943e8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 06:57:49 UTC) from {username='kudu', principal='kudu/127.19.52.126@KRBTEST.COM'} at 127.0.0.1:51732
I20260812 06:57:49.497431 3552 tablet_replica.cc:330] T 8d66008fbc174517b82ef15f30b943e8 P c25a0ae5f88b4400a21eb4c9085cd2c3: stopping tablet replica
I20260812 06:57:49.497845 3552 raft_consensus.cc:2229] T 8d66008fbc174517b82ef15f30b943e8 P c25a0ae5f88b4400a21eb4c9085cd2c3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 06:57:49.498270 3552 raft_consensus.cc:2258] T 8d66008fbc174517b82ef15f30b943e8 P c25a0ae5f88b4400a21eb4c9085cd2c3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 06:57:49.500218 3552 ts_tablet_manager.cc:1905] T 8d66008fbc174517b82ef15f30b943e8 P c25a0ae5f88b4400a21eb4c9085cd2c3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 06:57:49.500679 3553 tablet_replica.cc:330] T 8d66008fbc174517b82ef15f30b943e8 P acc217e1270e4e56ac27f8bbe6ed20ba: stopping tablet replica
I20260812 06:57:49.501116 3553 raft_consensus.cc:2229] T 8d66008fbc174517b82ef15f30b943e8 P acc217e1270e4e56ac27f8bbe6ed20ba [term 1 LEADER]: Raft consensus shutting down.
I20260812 06:57:49.501698 3553 raft_consensus.cc:2258] T 8d66008fbc174517b82ef15f30b943e8 P acc217e1270e4e56ac27f8bbe6ed20ba [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 06:57:49.503319 3553 ts_tablet_manager.cc:1905] T 8d66008fbc174517b82ef15f30b943e8 P acc217e1270e4e56ac27f8bbe6ed20ba: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 06:57:49.503701 3552 ts_tablet_manager.cc:1918] T 8d66008fbc174517b82ef15f30b943e8 P c25a0ae5f88b4400a21eb4c9085cd2c3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 06:57:49.503814 3552 log.cc:1196] T 8d66008fbc174517b82ef15f30b943e8 P c25a0ae5f88b4400a21eb4c9085cd2c3: Deleting WAL directory at /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517687384214-19665-0/minicluster-data/ts-2/wal/wals/8d66008fbc174517b82ef15f30b943e8
I20260812 06:57:49.504185 3552 ts_tablet_manager.cc:1939] T 8d66008fbc174517b82ef15f30b943e8 P c25a0ae5f88b4400a21eb4c9085cd2c3: Deleting consensus metadata
W20260812 06:57:49.505141 2958 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.19.52.65:43193 (error 108)
I20260812 06:57:49.505821 2961 catalog_manager.cc:4841] TS c25a0ae5f88b4400a21eb4c9085cd2c3 (127.19.52.67:46061): tablet 8d66008fbc174517b82ef15f30b943e8 (table test-table [id=fd66bc65982f4c809fb145056825e5f7]) successfully deleted
I20260812 06:57:49.506069 19665 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu with pid 3210
I20260812 06:57:49.509747 3523 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:57:49.506450 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.65:43193 (local address 127.0.0.1:51694)
0812 06:57:49.509576 (+ 3126us) negotiation.cc:107] Waiting for socket to connect
0812 06:57:49.509662 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.52.65:43193: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":3020}
W20260812 06:57:49.510797 2958 catalog_manager.cc:4584] TS 53890c7692834a24bd8919a9cbae000a (127.19.52.65:43193): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 8d66008fbc174517b82ef15f30b943e8: Network error: Client connection negotiation failed: client connection to 127.19.52.65:43193: connect: Connection refused (error 111)
I20260812 06:57:49.516121 19665 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu with pid 3364
I20260812 06:57:49.524155 3523 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:57:49.523718 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.66:36889 (local address 127.0.0.1:51750)
0812 06:57:49.523958 (+ 240us) negotiation.cc:107] Waiting for socket to connect
0812 06:57:49.524033 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.52.66:36889: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":136}
I20260812 06:57:49.529060 19665 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu with pid 2938
I20260812 06:57:49.538841 19665 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu
/tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517687384214-19665-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517687384214-19665-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517687384214-19665-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517687384214-19665-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.52.126:42629
--webserver_interface=127.19.52.126
--webserver_port=42859
--keytab_file=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517687384214-19665-0/krb5kdc/kudu_127.19.52.126.keytab
--principal=kudu/127.19.52.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.52.84:33591
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.19.52.126:42629
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--ipki_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517687384214-19665-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517687384214-19665-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517687384214-19665-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 06:57:49.662050 3555 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:57:49.662617 3555 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:57:49.662752 3555 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 06:57:49.668186 3555 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 06:57:49.668267 3555 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 06:57:49.668286 3555 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 06:57:49.668304 3555 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 06:57:49.668320 3555 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 06:57:49.674235 3555 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.52.84:33591
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517687384214-19665-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517687384214-19665-0/minicluster-data/master-0/wal
--ipki_private_key_password_cmd=echo foo
--ipki_ca_key_size=768
--master_addresses=127.19.52.126:42629
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517687384214-19665-0/krb5kdc/kudu_127.19.52.126.keytab
--principal=kudu/127.19.52.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.52.126:42629
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517687384214-19665-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.19.52.126
--webserver_port=42859
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3555
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517687384214-19665-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 06:15:54 UTC on ccycloud.quasar-rnfzce.root.comops.site
build id 4581989
I20260812 06:57:49.675411 3555 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:57:49.676440 3555 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 06:57:49.692931 3562 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:57:49.692937 3564 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:57:49.692971 3561 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:57:49.693193 3555 server_base.cc:1034] running on GCE node
I20260812 06:57:49.693990 3555 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:57:49.695410 3555 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:57:49.696640 3555 hybrid_clock.cc:648] HybridClock initialized: now 1786517869696621 us; error 32 us; skew 500 ppm
Aug 12 06:57:49 dist-test-slave-65mx krb5kdc[2923](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517869, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.52.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 06:57:49.699903 3555 init.cc:377] Logged in from keytab as kudu/127.19.52.126@KRBTEST.COM (short username kudu)
I20260812 06:57:49.700944 3555 webserver.cc:460] Webserver started at http://127.19.52.126:42859/ using document root <none> and password file <none>
I20260812 06:57:49.701531 3555 fs_manager.cc:359] Metadata directory not provided
I20260812 06:57:49.701612 3555 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:57:49.705861 3555 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260812 06:57:49.708379 3571 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 06:57:49.709931 3555 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 06:57:49.710057 3555 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517687384214-19665-0/minicluster-data/master-0/data,/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517687384214-19665-0/minicluster-data/master-0/wal
uuid: "cb9a783d342a46d4858bf6291d637d35"
format_stamp: "Formatted at 2026-08-12 06:57:48 on dist-test-slave-65mx"
server_key: "1165741c3419a9ea4d6292178791647f"
server_key_iv: "33a1a865a75e3f0cc43738e7bca493c4"
server_key_version: "encryptionkey@0"
I20260812 06:57:49.710767 3555 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517687384214-19665-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517687384214-19665-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517687384214-19665-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:57:49.735970 3555 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 06:57:49.739784 3555 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 06:57:49.740046 3555 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:57:49.748946 3555 rpc_server.cc:307] RPC server started. Bound to: 127.19.52.126:42629
I20260812 06:57:49.749373 3628 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.52.126:42629 every 8 connection(s)
I20260812 06:57:49.750442 3555 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517687384214-19665-0/minicluster-data/master-0/data/info.pb
I20260812 06:57:49.755335 3629 sys_catalog.cc:263] Verifying existing consensus state
I20260812 06:57:49.757321 19665 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu as pid 3555
I20260812 06:57:49.757439 3629 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cb9a783d342a46d4858bf6291d637d35: Bootstrap starting.
I20260812 06:57:49.757890 19665 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu
/tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517687384214-19665-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517687384214-19665-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517687384214-19665-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517687384214-19665-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.52.65:43193
--local_ip_for_outbound_sockets=127.19.52.65
--tserver_master_addrs=127.19.52.126:42629
--webserver_port=35641
--webserver_interface=127.19.52.65
--keytab_file=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517687384214-19665-0/krb5kdc/kudu_127.19.52.65.keytab
--principal=kudu/127.19.52.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.52.84:33591
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517687384214-19665-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517687384214-19665-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517687384214-19665-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 06:57:49.782639 3629 log.cc:824] T 00000000000000000000000000000000 P cb9a783d342a46d4858bf6291d637d35: Log is configured to *not* fsync() on all Append() calls
I20260812 06:57:49.795921 3629 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cb9a783d342a46d4858bf6291d637d35: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260812 06:57:49.796432 3629 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cb9a783d342a46d4858bf6291d637d35: Bootstrap complete.
I20260812 06:57:49.803103 3629 raft_consensus.cc:348] T 00000000000000000000000000000000 P cb9a783d342a46d4858bf6291d637d35 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cb9a783d342a46d4858bf6291d637d35" member_type: VOTER last_known_addr { host: "127.19.52.126" port: 42629 } }
I20260812 06:57:49.803711 3629 raft_consensus.cc:729] T 00000000000000000000000000000000 P cb9a783d342a46d4858bf6291d637d35 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: cb9a783d342a46d4858bf6291d637d35, State: Initialized, Role: FOLLOWER
I20260812 06:57:49.804332 3629 consensus_queue.cc:260] T 00000000000000000000000000000000 P cb9a783d342a46d4858bf6291d637d35 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cb9a783d342a46d4858bf6291d637d35" member_type: VOTER last_known_addr { host: "127.19.52.126" port: 42629 } }
I20260812 06:57:49.804651 3629 raft_consensus.cc:388] T 00000000000000000000000000000000 P cb9a783d342a46d4858bf6291d637d35 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 06:57:49.804754 3629 raft_consensus.cc:482] T 00000000000000000000000000000000 P cb9a783d342a46d4858bf6291d637d35 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 06:57:49.804867 3629 raft_consensus.cc:3037] T 00000000000000000000000000000000 P cb9a783d342a46d4858bf6291d637d35 [term 1 FOLLOWER]: Advancing to term 2
I20260812 06:57:49.808182 3629 raft_consensus.cc:504] T 00000000000000000000000000000000 P cb9a783d342a46d4858bf6291d637d35 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cb9a783d342a46d4858bf6291d637d35" member_type: VOTER last_known_addr { host: "127.19.52.126" port: 42629 } }
I20260812 06:57:49.808663 3629 leader_election.cc:302] T 00000000000000000000000000000000 P cb9a783d342a46d4858bf6291d637d35 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: cb9a783d342a46d4858bf6291d637d35; no voters:
I20260812 06:57:49.809067 3629 leader_election.cc:288] T 00000000000000000000000000000000 P cb9a783d342a46d4858bf6291d637d35 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260812 06:57:49.809366 3633 raft_consensus.cc:2781] T 00000000000000000000000000000000 P cb9a783d342a46d4858bf6291d637d35 [term 2 FOLLOWER]: Leader election won for term 2
I20260812 06:57:49.809691 3633 raft_consensus.cc:686] T 00000000000000000000000000000000 P cb9a783d342a46d4858bf6291d637d35 [term 2 LEADER]: Becoming Leader. State: Replica: cb9a783d342a46d4858bf6291d637d35, State: Running, Role: LEADER
I20260812 06:57:49.810034 3633 consensus_queue.cc:237] T 00000000000000000000000000000000 P cb9a783d342a46d4858bf6291d637d35 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cb9a783d342a46d4858bf6291d637d35" member_type: VOTER last_known_addr { host: "127.19.52.126" port: 42629 } }
I20260812 06:57:49.810494 3629 sys_catalog.cc:564] T 00000000000000000000000000000000 P cb9a783d342a46d4858bf6291d637d35 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 06:57:49.811667 3634 sys_catalog.cc:455] T 00000000000000000000000000000000 P cb9a783d342a46d4858bf6291d637d35 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "cb9a783d342a46d4858bf6291d637d35" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cb9a783d342a46d4858bf6291d637d35" member_type: VOTER last_known_addr { host: "127.19.52.126" port: 42629 } } }
I20260812 06:57:49.811807 3634 sys_catalog.cc:458] T 00000000000000000000000000000000 P cb9a783d342a46d4858bf6291d637d35 [sys.catalog]: This master's current role is: LEADER
I20260812 06:57:49.811869 3635 sys_catalog.cc:455] T 00000000000000000000000000000000 P cb9a783d342a46d4858bf6291d637d35 [sys.catalog]: SysCatalogTable state changed. Reason: New leader cb9a783d342a46d4858bf6291d637d35. Latest consensus state: current_term: 2 leader_uuid: "cb9a783d342a46d4858bf6291d637d35" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cb9a783d342a46d4858bf6291d637d35" member_type: VOTER last_known_addr { host: "127.19.52.126" port: 42629 } } }
I20260812 06:57:49.811949 3635 sys_catalog.cc:458] T 00000000000000000000000000000000 P cb9a783d342a46d4858bf6291d637d35 [sys.catalog]: This master's current role is: LEADER
I20260812 06:57:49.812333 3644 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 06:57:49.817832 3644 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 06:57:49.818773 3644 catalog_manager.cc:1259] Loaded cluster ID: e36be11f1aff48c080e5d4150585e637
I20260812 06:57:49.818861 3644 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 06:57:49.819994 3644 catalog_manager.cc:1314] IPKI private key was stored in cleartext, attempting to encrypt it.
I20260812 06:57:49.822865 3644 catalog_manager.cc:1324] IPKI private key successfully encrypted.
I20260812 06:57:49.827489 3644 catalog_manager.cc:1504] Loading token signing keys...
I20260812 06:57:49.828203 3644 catalog_manager.cc:5879] T 00000000000000000000000000000000 P cb9a783d342a46d4858bf6291d637d35: Loaded TSK: 0
I20260812 06:57:49.829289 3644 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260812 06:57:49.916652 3631 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:57:49.916967 3631 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:57:49.917059 3631 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 06:57:49.921849 3631 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 06:57:49.921936 3631 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 06:57:49.922003 3631 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.52.65
I20260812 06:57:49.927714 3631 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.52.84:33591
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517687384214-19665-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517687384214-19665-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517687384214-19665-0/krb5kdc/kudu_127.19.52.65.keytab
--principal=kudu/127.19.52.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.52.65:43193
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517687384214-19665-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.19.52.65
--webserver_port=35641
--tserver_master_addrs=127.19.52.126:42629
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3631
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.52.65
--log_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517687384214-19665-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 06:15:54 UTC on ccycloud.quasar-rnfzce.root.comops.site
build id 4581989
I20260812 06:57:49.929008 3631 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:57:49.930260 3631 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 06:57:49.940933 3661 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:57:49.940984 3658 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:57:49.940950 3659 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:57:49.941905 3631 server_base.cc:1034] running on GCE node
I20260812 06:57:49.942391 3631 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:57:49.943127 3631 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:57:49.944346 3631 hybrid_clock.cc:648] HybridClock initialized: now 1786517869944323 us; error 41 us; skew 500 ppm
Aug 12 06:57:49 dist-test-slave-65mx krb5kdc[2923](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517869, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.52.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 06:57:49.947974 3631 init.cc:377] Logged in from keytab as kudu/127.19.52.65@KRBTEST.COM (short username kudu)
I20260812 06:57:49.949183 3631 webserver.cc:460] Webserver started at http://127.19.52.65:35641/ using document root <none> and password file <none>
I20260812 06:57:49.949788 3631 fs_manager.cc:359] Metadata directory not provided
I20260812 06:57:49.949905 3631 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:57:49.954198 3631 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260812 06:57:49.956493 3669 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 06:57:49.957966 3631 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 06:57:49.958071 3631 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517687384214-19665-0/minicluster-data/ts-0/data,/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517687384214-19665-0/minicluster-data/ts-0/wal
uuid: "53890c7692834a24bd8919a9cbae000a"
format_stamp: "Formatted at 2026-08-12 06:57:48 on dist-test-slave-65mx"
server_key: "7ff113d789f532f2b56580096c1e3507"
server_key_iv: "4b60a721ba9c2d1bca1c96839ab5cf2e"
server_key_version: "encryptionkey@0"
I20260812 06:57:49.958668 3631 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517687384214-19665-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517687384214-19665-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517687384214-19665-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:57:49.996663 3631 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 06:57:49.999854 3631 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 06:57:50.000084 3631 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:57:50.000758 3631 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 06:57:50.001929 3676 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260812 06:57:50.006626 3631 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260812 06:57:50.006719 3631 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.005s user 0.000s sys 0.000s
I20260812 06:57:50.006800 3631 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260812 06:57:50.008570 3631 ts_tablet_manager.cc:610] Registered 1 tablets
I20260812 06:57:50.008653 3631 ts_tablet_manager.cc:589] Time spent register tablets: real 0.002s user 0.002s sys 0.000s
I20260812 06:57:50.008948 3676 tablet_bootstrap.cc:492] T 8d66008fbc174517b82ef15f30b943e8 P 53890c7692834a24bd8919a9cbae000a: Bootstrap starting.
I20260812 06:57:50.019716 3631 rpc_server.cc:307] RPC server started. Bound to: 127.19.52.65:43193
I20260812 06:57:50.019822 3798 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.52.65:43193 every 8 connection(s)
I20260812 06:57:50.020880 3631 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517687384214-19665-0/minicluster-data/ts-0/data/info.pb
I20260812 06:57:50.028340 19665 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu as pid 3631
I20260812 06:57:50.028870 19665 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu
/tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517687384214-19665-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517687384214-19665-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517687384214-19665-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517687384214-19665-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.52.66:36889
--local_ip_for_outbound_sockets=127.19.52.66
--tserver_master_addrs=127.19.52.126:42629
--webserver_port=34873
--webserver_interface=127.19.52.66
--keytab_file=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517687384214-19665-0/krb5kdc/kudu_127.19.52.66.keytab
--principal=kudu/127.19.52.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.52.84:33591
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517687384214-19665-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517687384214-19665-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517687384214-19665-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 06:57:50.030288 3676 log.cc:824] T 8d66008fbc174517b82ef15f30b943e8 P 53890c7692834a24bd8919a9cbae000a: Log is configured to *not* fsync() on all Append() calls
Aug 12 06:57:50 dist-test-slave-65mx krb5kdc[2923](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517869, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.52.65@KRBTEST.COM for kudu/127.19.52.126@KRBTEST.COM
I20260812 06:57:50.039608 3676 tablet_bootstrap.cc:492] T 8d66008fbc174517b82ef15f30b943e8 P 53890c7692834a24bd8919a9cbae000a: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260812 06:57:50.040123 3676 tablet_bootstrap.cc:492] T 8d66008fbc174517b82ef15f30b943e8 P 53890c7692834a24bd8919a9cbae000a: Bootstrap complete.
I20260812 06:57:50.041344 3802 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:57:50.023312 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.52.65:51641 (local address 127.19.52.126:42629)
0812 06:57:50.024815 (+ 1503us) server_negotiation.cc:206] Beginning negotiation
0812 06:57:50.024830 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0812 06:57:50.024868 (+ 38us) server_negotiation.cc:407] Connection header received
0812 06:57:50.025591 (+ 723us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:57:50.025601 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:57:50.025889 (+ 288us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:57:50.026193 (+ 304us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:57:50.027839 (+ 1646us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:57:50.029094 (+ 1255us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:57:50.030282 (+ 1188us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:57:50.030969 (+ 687us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:57:50.031171 (+ 202us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:57:50.034439 (+ 3268us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:57:50.034448 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:57:50.034466 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:57:50.034504 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:57:50.036957 (+ 2453us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:57:50.037712 (+ 755us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:57:50.037719 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:57:50.037724 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:57:50.037890 (+ 166us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:57:50.038359 (+ 469us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:57:50.038362 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:57:50.038364 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:57:50.038834 (+ 470us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:57:50.038972 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0812 06:57:50.039726 (+ 754us) server_negotiation.cc:299] Negotiation successful
0812 06:57:50.040406 (+ 680us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1382,"thread_start_us":166,"threads_started":1}
I20260812 06:57:50.041383 3803 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:57:50.023240 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.126:42629 (local address 127.19.52.65:51641)
0812 06:57:50.023816 (+ 576us) negotiation.cc:107] Waiting for socket to connect
0812 06:57:50.023883 (+ 67us) client_negotiation.cc:174] Beginning negotiation
0812 06:57:50.025194 (+ 1311us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:57:50.026359 (+ 1165us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:57:50.026371 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:57:50.026878 (+ 507us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:57:50.027607 (+ 729us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:57:50.027636 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:57:50.029263 (+ 1627us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:57:50.029266 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:57:50.030138 (+ 872us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:57:50.030148 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:57:50.031156 (+ 1008us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:57:50.031160 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:57:50.031367 (+ 207us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:57:50.032058 (+ 691us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 06:57:50.032079 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:57:50.034178 (+ 2099us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 06:57:50.037117 (+ 2939us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:57:50.037126 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:57:50.037143 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:57:50.037561 (+ 418us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:57:50.038013 (+ 452us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:57:50.038025 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:57:50.038027 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:57:50.038234 (+ 207us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:57:50.038980 (+ 746us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 06:57:50.038987 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 06:57:50.039342 (+ 355us) client_negotiation.cc:769] Sending connection context
0812 06:57:50.039580 (+ 238us) client_negotiation.cc:240] Negotiation successful
0812 06:57:50.040114 (+ 534us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":335,"thread_start_us":153,"threads_started":1}
I20260812 06:57:50.041738 3676 ts_tablet_manager.cc:1397] T 8d66008fbc174517b82ef15f30b943e8 P 53890c7692834a24bd8919a9cbae000a: Time spent bootstrapping tablet: real 0.033s user 0.019s sys 0.012s
I20260812 06:57:50.044495 3800 heartbeater.cc:347] Connected to a master server at 127.19.52.126:42629
I20260812 06:57:50.044785 3800 heartbeater.cc:464] Registering TS with master...
I20260812 06:57:50.044569 3676 raft_consensus.cc:348] T 8d66008fbc174517b82ef15f30b943e8 P 53890c7692834a24bd8919a9cbae000a [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acc217e1270e4e56ac27f8bbe6ed20ba" member_type: VOTER last_known_addr { host: "127.19.52.66" port: 36889 } } peers { permanent_uuid: "c25a0ae5f88b4400a21eb4c9085cd2c3" member_type: VOTER last_known_addr { host: "127.19.52.67" port: 46061 } } peers { permanent_uuid: "53890c7692834a24bd8919a9cbae000a" member_type: VOTER last_known_addr { host: "127.19.52.65" port: 43193 } }
I20260812 06:57:50.045113 3676 raft_consensus.cc:729] T 8d66008fbc174517b82ef15f30b943e8 P 53890c7692834a24bd8919a9cbae000a [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 53890c7692834a24bd8919a9cbae000a, State: Initialized, Role: FOLLOWER
I20260812 06:57:50.045353 3800 heartbeater.cc:511] Master 127.19.52.126:42629 requested a full tablet report, sending...
I20260812 06:57:50.045603 3676 consensus_queue.cc:260] T 8d66008fbc174517b82ef15f30b943e8 P 53890c7692834a24bd8919a9cbae000a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acc217e1270e4e56ac27f8bbe6ed20ba" member_type: VOTER last_known_addr { host: "127.19.52.66" port: 36889 } } peers { permanent_uuid: "c25a0ae5f88b4400a21eb4c9085cd2c3" member_type: VOTER last_known_addr { host: "127.19.52.67" port: 46061 } } peers { permanent_uuid: "53890c7692834a24bd8919a9cbae000a" member_type: VOTER last_known_addr { host: "127.19.52.65" port: 43193 } }
I20260812 06:57:50.046684 3676 ts_tablet_manager.cc:1428] T 8d66008fbc174517b82ef15f30b943e8 P 53890c7692834a24bd8919a9cbae000a: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260812 06:57:50.048148 3590 ts_manager.cc:194] Registered new tserver with Master: 53890c7692834a24bd8919a9cbae000a (127.19.52.65:43193)
I20260812 06:57:50.051587 3590 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.52.65@KRBTEST.COM'} at 127.19.52.65:51641
I20260812 06:57:50.054242 3800 heartbeater.cc:503] Master 127.19.52.126:42629 was elected leader, sending a full tablet report...
Aug 12 06:57:50 dist-test-slave-65mx krb5kdc[2923](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517869, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.52.126@KRBTEST.COM for kudu/127.19.52.65@KRBTEST.COM
I20260812 06:57:50.062213 3810 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:57:50.051101 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.65:43193 (local address 127.0.0.1:51704)
0812 06:57:50.051537 (+ 436us) negotiation.cc:107] Waiting for socket to connect
0812 06:57:50.051582 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0812 06:57:50.051718 (+ 136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:57:50.052445 (+ 727us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:57:50.052450 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:57:50.052855 (+ 405us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:57:50.052959 (+ 104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:57:50.052966 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:57:50.053941 (+ 975us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:57:50.053946 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:57:50.054695 (+ 749us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:57:50.054707 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:57:50.055519 (+ 812us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:57:50.055524 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:57:50.055636 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:57:50.056429 (+ 793us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 06:57:50.056444 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:57:50.058015 (+ 1571us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 06:57:50.060057 (+ 2042us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:57:50.060067 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:57:50.060073 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:57:50.060317 (+ 244us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:57:50.060710 (+ 393us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:57:50.060714 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:57:50.060716 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:57:50.060810 (+ 94us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:57:50.061302 (+ 492us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 06:57:50.061308 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 06:57:50.061551 (+ 243us) client_negotiation.cc:769] Sending connection context
0812 06:57:50.061755 (+ 204us) client_negotiation.cc:240] Negotiation successful
0812 06:57:50.062006 (+ 251us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":254,"thread_start_us":139,"threads_started":1}
I20260812 06:57:50.062248 3811 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:57:50.051226 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51704 (local address 127.19.52.65:43193)
0812 06:57:50.051956 (+ 730us) server_negotiation.cc:206] Beginning negotiation
0812 06:57:50.051967 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0812 06:57:50.051991 (+ 24us) server_negotiation.cc:407] Connection header received
0812 06:57:50.052075 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:57:50.052080 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:57:50.052274 (+ 194us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:57:50.052406 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:57:50.053103 (+ 697us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:57:50.053813 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:57:50.054828 (+ 1015us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:57:50.055378 (+ 550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:57:50.055492 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:57:50.058166 (+ 2674us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:57:50.058183 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:57:50.058193 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:57:50.058215 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:57:50.059880 (+ 1665us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:57:50.060448 (+ 568us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:57:50.060459 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:57:50.060460 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:57:50.060599 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:57:50.060909 (+ 310us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:57:50.060913 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:57:50.060915 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:57:50.061176 (+ 261us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:57:50.061326 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0812 06:57:50.061867 (+ 541us) server_negotiation.cc:299] Negotiation successful
0812 06:57:50.062079 (+ 212us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":633,"thread_start_us":97,"threads_started":1}
I20260812 06:57:50.063529 3727 tablet_service.cc:1514] Processing DeleteTablet for tablet 8d66008fbc174517b82ef15f30b943e8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 06:57:49 UTC) from {username='kudu', principal='kudu/127.19.52.126@KRBTEST.COM'} at 127.0.0.1:51704
I20260812 06:57:50.064507 3812 tablet_replica.cc:330] stopping tablet replica
I20260812 06:57:50.064860 3812 raft_consensus.cc:2229] T 8d66008fbc174517b82ef15f30b943e8 P 53890c7692834a24bd8919a9cbae000a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 06:57:50.065133 3812 raft_consensus.cc:2258] T 8d66008fbc174517b82ef15f30b943e8 P 53890c7692834a24bd8919a9cbae000a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 06:57:50.066339 3812 ts_tablet_manager.cc:1905] T 8d66008fbc174517b82ef15f30b943e8 P 53890c7692834a24bd8919a9cbae000a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 06:57:50.071161 3812 ts_tablet_manager.cc:1918] T 8d66008fbc174517b82ef15f30b943e8 P 53890c7692834a24bd8919a9cbae000a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 06:57:50.071283 3812 log.cc:1196] T 8d66008fbc174517b82ef15f30b943e8 P 53890c7692834a24bd8919a9cbae000a: Deleting WAL directory at /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517687384214-19665-0/minicluster-data/ts-0/wal/wals/8d66008fbc174517b82ef15f30b943e8
I20260812 06:57:50.071578 3812 ts_tablet_manager.cc:1939] T 8d66008fbc174517b82ef15f30b943e8 P 53890c7692834a24bd8919a9cbae000a: Deleting consensus metadata
I20260812 06:57:50.073745 3572 catalog_manager.cc:4841] TS 53890c7692834a24bd8919a9cbae000a (127.19.52.65:43193): tablet 8d66008fbc174517b82ef15f30b943e8 (table test-table [id=fd66bc65982f4c809fb145056825e5f7]) successfully deleted
W20260812 06:57:50.157794 3804 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:57:50.158087 3804 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:57:50.158139 3804 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 06:57:50.162286 3804 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 06:57:50.162412 3804 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 06:57:50.162480 3804 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.52.66
I20260812 06:57:50.167999 3804 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.52.84:33591
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517687384214-19665-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517687384214-19665-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517687384214-19665-0/krb5kdc/kudu_127.19.52.66.keytab
--principal=kudu/127.19.52.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.52.66:36889
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517687384214-19665-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.19.52.66
--webserver_port=34873
--tserver_master_addrs=127.19.52.126:42629
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3804
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.52.66
--log_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517687384214-19665-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 06:15:54 UTC on ccycloud.quasar-rnfzce.root.comops.site
build id 4581989
I20260812 06:57:50.169127 3804 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:57:50.170120 3804 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 06:57:50.180130 3817 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:57:50.180260 3822 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:57:50.180240 3804 server_base.cc:1034] running on GCE node
W20260812 06:57:50.180133 3818 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:57:50.180948 3804 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:57:50.181633 3804 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:57:50.182832 3804 hybrid_clock.cc:648] HybridClock initialized: now 1786517870182814 us; error 30 us; skew 500 ppm
Aug 12 06:57:50 dist-test-slave-65mx krb5kdc[2923](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517870, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.52.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 06:57:50.185850 3804 init.cc:377] Logged in from keytab as kudu/127.19.52.66@KRBTEST.COM (short username kudu)
I20260812 06:57:50.186729 3804 webserver.cc:460] Webserver started at http://127.19.52.66:34873/ using document root <none> and password file <none>
I20260812 06:57:50.187270 3804 fs_manager.cc:359] Metadata directory not provided
I20260812 06:57:50.187345 3804 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:57:50.191197 3804 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.002s sys 0.000s
I20260812 06:57:50.193539 3829 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 06:57:50.194905 3804 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 06:57:50.195070 3804 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517687384214-19665-0/minicluster-data/ts-1/data,/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517687384214-19665-0/minicluster-data/ts-1/wal
uuid: "acc217e1270e4e56ac27f8bbe6ed20ba"
format_stamp: "Formatted at 2026-08-12 06:57:48 on dist-test-slave-65mx"
server_key: "1dd0239db6827471cbbe582197123db3"
server_key_iv: "2427b2eef86a471923318659774f55fb"
server_key_version: "encryptionkey@0"
I20260812 06:57:50.195632 3804 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517687384214-19665-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517687384214-19665-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517687384214-19665-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:57:50.208132 3804 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 06:57:50.211338 3804 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 06:57:50.211618 3804 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:57:50.212328 3804 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 06:57:50.213577 3837 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260812 06:57:50.218109 3804 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260812 06:57:50.218215 3804 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.005s user 0.000s sys 0.000s
I20260812 06:57:50.218343 3804 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260812 06:57:50.220077 3804 ts_tablet_manager.cc:610] Registered 1 tablets
I20260812 06:57:50.220161 3804 ts_tablet_manager.cc:589] Time spent register tablets: real 0.002s user 0.000s sys 0.000s
I20260812 06:57:50.220405 3837 tablet_bootstrap.cc:492] T 8d66008fbc174517b82ef15f30b943e8 P acc217e1270e4e56ac27f8bbe6ed20ba: Bootstrap starting.
I20260812 06:57:50.226428 3837 log.cc:824] T 8d66008fbc174517b82ef15f30b943e8 P acc217e1270e4e56ac27f8bbe6ed20ba: Log is configured to *not* fsync() on all Append() calls
I20260812 06:57:50.231046 3804 rpc_server.cc:307] RPC server started. Bound to: 127.19.52.66:36889
I20260812 06:57:50.231108 3963 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.52.66:36889 every 8 connection(s)
I20260812 06:57:50.231966 3804 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517687384214-19665-0/minicluster-data/ts-1/data/info.pb
I20260812 06:57:50.235311 3837 tablet_bootstrap.cc:492] T 8d66008fbc174517b82ef15f30b943e8 P acc217e1270e4e56ac27f8bbe6ed20ba: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260812 06:57:50.235801 3837 tablet_bootstrap.cc:492] T 8d66008fbc174517b82ef15f30b943e8 P acc217e1270e4e56ac27f8bbe6ed20ba: Bootstrap complete.
I20260812 06:57:50.236183 3837 ts_tablet_manager.cc:1397] T 8d66008fbc174517b82ef15f30b943e8 P acc217e1270e4e56ac27f8bbe6ed20ba: Time spent bootstrapping tablet: real 0.016s user 0.012s sys 0.000s
I20260812 06:57:50.239326 19665 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu as pid 3804
I20260812 06:57:50.239199 3837 raft_consensus.cc:348] T 8d66008fbc174517b82ef15f30b943e8 P acc217e1270e4e56ac27f8bbe6ed20ba [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acc217e1270e4e56ac27f8bbe6ed20ba" member_type: VOTER last_known_addr { host: "127.19.52.66" port: 36889 } } peers { permanent_uuid: "c25a0ae5f88b4400a21eb4c9085cd2c3" member_type: VOTER last_known_addr { host: "127.19.52.67" port: 46061 } } peers { permanent_uuid: "53890c7692834a24bd8919a9cbae000a" member_type: VOTER last_known_addr { host: "127.19.52.65" port: 43193 } }
I20260812 06:57:50.239727 3837 raft_consensus.cc:729] T 8d66008fbc174517b82ef15f30b943e8 P acc217e1270e4e56ac27f8bbe6ed20ba [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: acc217e1270e4e56ac27f8bbe6ed20ba, State: Initialized, Role: FOLLOWER
I20260812 06:57:50.239831 19665 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu
/tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517687384214-19665-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517687384214-19665-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517687384214-19665-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517687384214-19665-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.52.67:46061
--local_ip_for_outbound_sockets=127.19.52.67
--tserver_master_addrs=127.19.52.126:42629
--webserver_port=42985
--webserver_interface=127.19.52.67
--keytab_file=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517687384214-19665-0/krb5kdc/kudu_127.19.52.67.keytab
--principal=kudu/127.19.52.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.52.84:33591
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517687384214-19665-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517687384214-19665-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517687384214-19665-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 06:57:50.240284 3837 consensus_queue.cc:260] T 8d66008fbc174517b82ef15f30b943e8 P acc217e1270e4e56ac27f8bbe6ed20ba [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acc217e1270e4e56ac27f8bbe6ed20ba" member_type: VOTER last_known_addr { host: "127.19.52.66" port: 36889 } } peers { permanent_uuid: "c25a0ae5f88b4400a21eb4c9085cd2c3" member_type: VOTER last_known_addr { host: "127.19.52.67" port: 46061 } } peers { permanent_uuid: "53890c7692834a24bd8919a9cbae000a" member_type: VOTER last_known_addr { host: "127.19.52.65" port: 43193 } }
I20260812 06:57:50.241442 3837 ts_tablet_manager.cc:1428] T 8d66008fbc174517b82ef15f30b943e8 P acc217e1270e4e56ac27f8bbe6ed20ba: Time spent starting tablet: real 0.005s user 0.007s sys 0.001s
Aug 12 06:57:50 dist-test-slave-65mx krb5kdc[2923](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517870, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.52.66@KRBTEST.COM for kudu/127.19.52.126@KRBTEST.COM
I20260812 06:57:50.248906 3802 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:57:50.233997 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.52.66:59833 (local address 127.19.52.126:42629)
0812 06:57:50.234171 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0812 06:57:50.234177 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 06:57:50.234879 (+ 702us) server_negotiation.cc:407] Connection header received
0812 06:57:50.236130 (+ 1251us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:57:50.236134 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:57:50.236183 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:57:50.236266 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:57:50.237782 (+ 1516us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:57:50.238310 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:57:50.239959 (+ 1649us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:57:50.240319 (+ 360us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:57:50.240436 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:57:50.244069 (+ 3633us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:57:50.244073 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:57:50.244075 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:57:50.244101 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:57:50.246133 (+ 2032us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:57:50.246783 (+ 650us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:57:50.246790 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:57:50.246794 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:57:50.246902 (+ 108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:57:50.247404 (+ 502us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:57:50.247408 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:57:50.247412 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:57:50.247730 (+ 318us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:57:50.247852 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0812 06:57:50.248436 (+ 584us) server_negotiation.cc:299] Negotiation successful
0812 06:57:50.248654 (+ 218us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260812 06:57:50.249716 3968 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:57:50.234098 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.126:42629 (local address 127.19.52.66:59833)
0812 06:57:50.234699 (+ 601us) negotiation.cc:107] Waiting for socket to connect
0812 06:57:50.234753 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0812 06:57:50.235872 (+ 1119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:57:50.236462 (+ 590us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:57:50.236471 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:57:50.236987 (+ 516us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:57:50.237612 (+ 625us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:57:50.237632 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:57:50.238465 (+ 833us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:57:50.238472 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:57:50.239756 (+ 1284us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:57:50.239767 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:57:50.240607 (+ 840us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:57:50.240614 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:57:50.240813 (+ 199us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:57:50.242065 (+ 1252us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 06:57:50.242092 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:57:50.243852 (+ 1760us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 06:57:50.246304 (+ 2452us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:57:50.246310 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:57:50.246324 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:57:50.246657 (+ 333us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:57:50.247078 (+ 421us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:57:50.247081 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:57:50.247083 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:57:50.247264 (+ 181us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:57:50.247871 (+ 607us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 06:57:50.247877 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 06:57:50.248158 (+ 281us) client_negotiation.cc:769] Sending connection context
0812 06:57:50.248342 (+ 184us) client_negotiation.cc:240] Negotiation successful
0812 06:57:50.248738 (+ 396us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":297,"thread_start_us":141,"threads_started":1}
I20260812 06:57:50.251080 3965 heartbeater.cc:347] Connected to a master server at 127.19.52.126:42629
I20260812 06:57:50.251345 3965 heartbeater.cc:464] Registering TS with master...
I20260812 06:57:50.251857 3965 heartbeater.cc:511] Master 127.19.52.126:42629 requested a full tablet report, sending...
I20260812 06:57:50.253669 3590 ts_manager.cc:194] Registered new tserver with Master: acc217e1270e4e56ac27f8bbe6ed20ba (127.19.52.66:36889)
I20260812 06:57:50.254918 3590 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.52.66@KRBTEST.COM'} at 127.19.52.66:59833
I20260812 06:57:50.257193 3965 heartbeater.cc:503] Master 127.19.52.126:42629 was elected leader, sending a full tablet report...
Aug 12 06:57:50 dist-test-slave-65mx krb5kdc[2923](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517869, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.52.126@KRBTEST.COM for kudu/127.19.52.66@KRBTEST.COM
I20260812 06:57:50.265657 3810 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:57:50.254587 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.66:36889 (local address 127.0.0.1:51752)
0812 06:57:50.254754 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0812 06:57:50.254769 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 06:57:50.254853 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:57:50.255658 (+ 805us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:57:50.255661 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:57:50.255874 (+ 213us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:57:50.255978 (+ 104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:57:50.255985 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:57:50.257520 (+ 1535us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:57:50.257523 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:57:50.258174 (+ 651us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:57:50.258183 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:57:50.258936 (+ 753us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:57:50.258940 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:57:50.259045 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:57:50.259556 (+ 511us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 06:57:50.259575 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:57:50.261232 (+ 1657us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 06:57:50.263838 (+ 2606us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:57:50.263843 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:57:50.263847 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:57:50.264113 (+ 266us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:57:50.264425 (+ 312us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:57:50.264429 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:57:50.264431 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:57:50.264534 (+ 103us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:57:50.265086 (+ 552us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 06:57:50.265089 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 06:57:50.265205 (+ 116us) client_negotiation.cc:769] Sending connection context
0812 06:57:50.265317 (+ 112us) client_negotiation.cc:240] Negotiation successful
0812 06:57:50.265479 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":79}
I20260812 06:57:50.265952 3971 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:57:50.254804 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51752 (local address 127.19.52.66:36889)
0812 06:57:50.255183 (+ 379us) server_negotiation.cc:206] Beginning negotiation
0812 06:57:50.255194 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0812 06:57:50.255216 (+ 22us) server_negotiation.cc:407] Connection header received
0812 06:57:50.255281 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:57:50.255286 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:57:50.255495 (+ 209us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:57:50.255640 (+ 145us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:57:50.256111 (+ 471us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:57:50.257385 (+ 1274us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:57:50.258302 (+ 917us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:57:50.258810 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:57:50.258925 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:57:50.261423 (+ 2498us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:57:50.261433 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:57:50.261473 (+ 40us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:57:50.261499 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:57:50.263720 (+ 2221us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:57:50.264235 (+ 515us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:57:50.264241 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:57:50.264242 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:57:50.264327 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:57:50.264688 (+ 361us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:57:50.264691 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:57:50.264692 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:57:50.264972 (+ 280us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:57:50.265091 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0812 06:57:50.265396 (+ 305us) server_negotiation.cc:299] Negotiation successful
0812 06:57:50.265624 (+ 228us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":6,"server-negotiator.queue_time_us":282,"thread_start_us":164,"threads_started":1}
I20260812 06:57:50.267030 3882 tablet_service.cc:1514] Processing DeleteTablet for tablet 8d66008fbc174517b82ef15f30b943e8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 06:57:49 UTC) from {username='kudu', principal='kudu/127.19.52.126@KRBTEST.COM'} at 127.0.0.1:51752
I20260812 06:57:50.268162 3973 tablet_replica.cc:330] stopping tablet replica
I20260812 06:57:50.268493 3973 raft_consensus.cc:2229] T 8d66008fbc174517b82ef15f30b943e8 P acc217e1270e4e56ac27f8bbe6ed20ba [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 06:57:50.268777 3973 raft_consensus.cc:2258] T 8d66008fbc174517b82ef15f30b943e8 P acc217e1270e4e56ac27f8bbe6ed20ba [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 06:57:50.269765 3973 ts_tablet_manager.cc:1905] T 8d66008fbc174517b82ef15f30b943e8 P acc217e1270e4e56ac27f8bbe6ed20ba: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 06:57:50.273679 3973 ts_tablet_manager.cc:1918] T 8d66008fbc174517b82ef15f30b943e8 P acc217e1270e4e56ac27f8bbe6ed20ba: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 06:57:50.273821 3973 log.cc:1196] T 8d66008fbc174517b82ef15f30b943e8 P acc217e1270e4e56ac27f8bbe6ed20ba: Deleting WAL directory at /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517687384214-19665-0/minicluster-data/ts-1/wal/wals/8d66008fbc174517b82ef15f30b943e8
I20260812 06:57:50.274149 3973 ts_tablet_manager.cc:1939] T 8d66008fbc174517b82ef15f30b943e8 P acc217e1270e4e56ac27f8bbe6ed20ba: Deleting consensus metadata
I20260812 06:57:50.275985 3572 catalog_manager.cc:4841] TS acc217e1270e4e56ac27f8bbe6ed20ba (127.19.52.66:36889): tablet 8d66008fbc174517b82ef15f30b943e8 (table test-table [id=fd66bc65982f4c809fb145056825e5f7]) successfully deleted
W20260812 06:57:50.365388 3970 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:57:50.365716 3970 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:57:50.365796 3970 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 06:57:50.370131 3970 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 06:57:50.370236 3970 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 06:57:50.370334 3970 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.52.67
I20260812 06:57:50.376695 3970 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.52.84:33591
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517687384214-19665-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517687384214-19665-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517687384214-19665-0/krb5kdc/kudu_127.19.52.67.keytab
--principal=kudu/127.19.52.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.52.67:46061
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517687384214-19665-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.19.52.67
--webserver_port=42985
--tserver_master_addrs=127.19.52.126:42629
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3970
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.52.67
--log_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517687384214-19665-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 06:15:54 UTC on ccycloud.quasar-rnfzce.root.comops.site
build id 4581989
I20260812 06:57:50.377964 3970 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:57:50.379065 3970 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 06:57:50.389592 3979 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:57:50.389669 3970 server_base.cc:1034] running on GCE node
W20260812 06:57:50.389595 3981 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:57:50.389870 3978 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:57:50.390281 3970 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:57:50.390983 3970 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:57:50.392197 3970 hybrid_clock.cc:648] HybridClock initialized: now 1786517870392161 us; error 45 us; skew 500 ppm
Aug 12 06:57:50 dist-test-slave-65mx krb5kdc[2923](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517870, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.52.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 06:57:50.395272 3970 init.cc:377] Logged in from keytab as kudu/127.19.52.67@KRBTEST.COM (short username kudu)
I20260812 06:57:50.396193 3970 webserver.cc:460] Webserver started at http://127.19.52.67:42985/ using document root <none> and password file <none>
I20260812 06:57:50.396880 3970 fs_manager.cc:359] Metadata directory not provided
I20260812 06:57:50.397002 3970 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:57:50.401374 3970 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.000s sys 0.003s
I20260812 06:57:50.403560 3988 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 06:57:50.405017 3970 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260812 06:57:50.405161 3970 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517687384214-19665-0/minicluster-data/ts-2/data,/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517687384214-19665-0/minicluster-data/ts-2/wal
uuid: "c25a0ae5f88b4400a21eb4c9085cd2c3"
format_stamp: "Formatted at 2026-08-12 06:57:49 on dist-test-slave-65mx"
server_key: "87c1d3b0e65710d14c7e34951a9c243f"
server_key_iv: "1df5e0e999d968e6721cdfc759ff71bc"
server_key_version: "encryptionkey@0"
I20260812 06:57:50.405752 3970 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517687384214-19665-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517687384214-19665-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517687384214-19665-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:57:50.452286 3970 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 06:57:50.455938 3970 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 06:57:50.456179 3970 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:57:50.456868 3970 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 06:57:50.457780 3970 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 06:57:50.457862 3970 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 06:57:50.457940 3970 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 06:57:50.457988 3970 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 06:57:50.467938 3970 rpc_server.cc:307] RPC server started. Bound to: 127.19.52.67:46061
I20260812 06:57:50.467978 4116 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.52.67:46061 every 8 connection(s)
I20260812 06:57:50.469065 3970 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517687384214-19665-0/minicluster-data/ts-2/data/info.pb
I20260812 06:57:50.469535 19665 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu as pid 3970
I20260812 06:57:50.479650 3802 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:57:50.470418 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44388 (local address 127.19.52.126:42629)
0812 06:57:50.470665 (+ 247us) server_negotiation.cc:206] Beginning negotiation
0812 06:57:50.470670 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 06:57:50.470840 (+ 170us) server_negotiation.cc:407] Connection header received
0812 06:57:50.471016 (+ 176us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:57:50.471020 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:57:50.471074 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:57:50.471171 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:57:50.472128 (+ 957us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:57:50.472891 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:57:50.473881 (+ 990us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:57:50.474237 (+ 356us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:57:50.474348 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:57:50.475909 (+ 1561us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:57:50.475913 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:57:50.475915 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:57:50.475942 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:57:50.477821 (+ 1879us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:57:50.478273 (+ 452us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:57:50.478277 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:57:50.478278 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:57:50.478374 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:57:50.478683 (+ 309us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:57:50.478686 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:57:50.478687 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:57:50.478864 (+ 177us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:57:50.479082 (+ 218us) server_negotiation.cc:1035] Waiting for connection context
0812 06:57:50.479306 (+ 224us) server_negotiation.cc:299] Negotiation successful
0812 06:57:50.479429 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":132}
Aug 12 06:57:50 dist-test-slave-65mx krb5kdc[2923](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517870, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.52.67@KRBTEST.COM for kudu/127.19.52.126@KRBTEST.COM
I20260812 06:57:50.485417 4120 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:57:50.471644 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.52.67:33163 (local address 127.19.52.126:42629)
0812 06:57:50.471891 (+ 247us) server_negotiation.cc:206] Beginning negotiation
0812 06:57:50.471895 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 06:57:50.472860 (+ 965us) server_negotiation.cc:407] Connection header received
0812 06:57:50.474457 (+ 1597us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:57:50.474460 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:57:50.474515 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:57:50.474625 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:57:50.476127 (+ 1502us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:57:50.476945 (+ 818us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:57:50.478633 (+ 1688us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:57:50.478891 (+ 258us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:57:50.479024 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:57:50.481875 (+ 2851us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:57:50.481878 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:57:50.481880 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:57:50.481901 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:57:50.483248 (+ 1347us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:57:50.483800 (+ 552us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:57:50.483803 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:57:50.483804 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:57:50.483886 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:57:50.484210 (+ 324us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:57:50.484212 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:57:50.484213 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:57:50.484441 (+ 228us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:57:50.484519 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0812 06:57:50.485156 (+ 637us) server_negotiation.cc:299] Negotiation successful
0812 06:57:50.485279 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":172,"thread_start_us":74,"threads_started":1}
I20260812 06:57:50.486222 4121 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:57:50.472104 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.126:42629 (local address 127.19.52.67:33163)
0812 06:57:50.472673 (+ 569us) negotiation.cc:107] Waiting for socket to connect
0812 06:57:50.472733 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0812 06:57:50.474217 (+ 1484us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:57:50.474920 (+ 703us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:57:50.474931 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:57:50.475385 (+ 454us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:57:50.475960 (+ 575us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:57:50.475982 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:57:50.477088 (+ 1106us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:57:50.477110 (+ 22us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:57:50.478505 (+ 1395us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:57:50.478514 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:57:50.479094 (+ 580us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:57:50.479101 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:57:50.479330 (+ 229us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:57:50.480038 (+ 708us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 06:57:50.480058 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:57:50.481741 (+ 1683us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 06:57:50.483380 (+ 1639us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:57:50.483384 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:57:50.483397 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:57:50.483694 (+ 297us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:57:50.483978 (+ 284us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:57:50.483980 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:57:50.483981 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:57:50.484122 (+ 141us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:57:50.484615 (+ 493us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 06:57:50.484625 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 06:57:50.484882 (+ 257us) client_negotiation.cc:769] Sending connection context
0812 06:57:50.485056 (+ 174us) client_negotiation.cc:240] Negotiation successful
0812 06:57:50.485329 (+ 273us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":312,"thread_start_us":132,"threads_started":1}
I20260812 06:57:50.487534 4117 heartbeater.cc:347] Connected to a master server at 127.19.52.126:42629
I20260812 06:57:50.487835 4117 heartbeater.cc:464] Registering TS with master...
I20260812 06:57:50.488442 4117 heartbeater.cc:511] Master 127.19.52.126:42629 requested a full tablet report, sending...
I20260812 06:57:50.489641 3590 ts_manager.cc:194] Registered new tserver with Master: c25a0ae5f88b4400a21eb4c9085cd2c3 (127.19.52.67:46061)
I20260812 06:57:50.490255 3590 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.52.67@KRBTEST.COM'} at 127.19.52.67:33163
I20260812 06:57:50.493546 19665 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 06:57:50.498507 4120 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:57:50.494238 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44396 (local address 127.19.52.126:42629)
0812 06:57:50.494364 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0812 06:57:50.494368 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 06:57:50.494621 (+ 253us) server_negotiation.cc:407] Connection header received
0812 06:57:50.494726 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:57:50.494729 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:57:50.494857 (+ 128us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:57:50.494990 (+ 133us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 06:57:50.495244 (+ 254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:57:50.495769 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:57:50.496890 (+ 1121us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:57:50.497166 (+ 276us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:57:50.497299 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:57:50.497657 (+ 358us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 06:57:50.498075 (+ 418us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 06:57:50.498217 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0812 06:57:50.498275 (+ 58us) server_negotiation.cc:299] Negotiation successful
0812 06:57:50.498344 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260812 06:57:50.499222 3590 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin'} at 127.0.0.1:44396:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 06:57:50.502795 3590 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 06:57:50.511096 3882 tablet_service.cc:1467] Processing CreateTablet for tablet ae36909a86394e4fada3c2cee01b3bb8 (DEFAULT_TABLE table=test-table [id=36c00697b24c43e9b9ac20a6b3f4012e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 06:57:50.511862 3882 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ae36909a86394e4fada3c2cee01b3bb8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 06:57:50.511962 3727 tablet_service.cc:1467] Processing CreateTablet for tablet ae36909a86394e4fada3c2cee01b3bb8 (DEFAULT_TABLE table=test-table [id=36c00697b24c43e9b9ac20a6b3f4012e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 06:57:50.512588 3727 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ae36909a86394e4fada3c2cee01b3bb8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 06:57:50.515027 3837 tablet_bootstrap.cc:492] T ae36909a86394e4fada3c2cee01b3bb8 P acc217e1270e4e56ac27f8bbe6ed20ba: Bootstrap starting.
I20260812 06:57:50.515470 3810 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:57:50.510181 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.67:46061 (local address 127.0.0.1:44708)
0812 06:57:50.510339 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0812 06:57:50.510355 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0812 06:57:50.510462 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:57:50.511330 (+ 868us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:57:50.511336 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:57:50.511360 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 06:57:50.511491 (+ 131us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:57:50.511498 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:57:50.513025 (+ 1527us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:57:50.513029 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:57:50.513808 (+ 779us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:57:50.513816 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:57:50.514914 (+ 1098us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:57:50.514918 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:57:50.515048 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:57:50.515103 (+ 55us) client_negotiation.cc:769] Sending connection context
0812 06:57:50.515212 (+ 109us) client_negotiation.cc:240] Negotiation successful
0812 06:57:50.515300 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":77}
I20260812 06:57:50.515496 4127 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:57:50.510303 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44708 (local address 127.19.52.67:46061)
0812 06:57:50.510738 (+ 435us) server_negotiation.cc:206] Beginning negotiation
0812 06:57:50.510751 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0812 06:57:50.510775 (+ 24us) server_negotiation.cc:407] Connection header received
0812 06:57:50.510854 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:57:50.510859 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:57:50.511097 (+ 238us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:57:50.511340 (+ 243us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 06:57:50.511630 (+ 290us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:57:50.512841 (+ 1211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:57:50.513947 (+ 1106us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:57:50.514773 (+ 826us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:57:50.514911 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:57:50.515066 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0812 06:57:50.515247 (+ 181us) server_negotiation.cc:299] Negotiation successful
0812 06:57:50.515342 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":324,"thread_start_us":144,"threads_started":1}
I20260812 06:57:50.515558 3676 tablet_bootstrap.cc:492] T ae36909a86394e4fada3c2cee01b3bb8 P 53890c7692834a24bd8919a9cbae000a: Bootstrap starting.
I20260812 06:57:50.516105 3837 tablet_bootstrap.cc:654] T ae36909a86394e4fada3c2cee01b3bb8 P acc217e1270e4e56ac27f8bbe6ed20ba: Neither blocks nor log segments found. Creating new log.
I20260812 06:57:50.516451 3676 tablet_bootstrap.cc:654] T ae36909a86394e4fada3c2cee01b3bb8 P 53890c7692834a24bd8919a9cbae000a: Neither blocks nor log segments found. Creating new log.
I20260812 06:57:50.518013 3837 tablet_bootstrap.cc:492] T ae36909a86394e4fada3c2cee01b3bb8 P acc217e1270e4e56ac27f8bbe6ed20ba: No bootstrap required, opened a new log
I20260812 06:57:50.518107 3837 ts_tablet_manager.cc:1397] T ae36909a86394e4fada3c2cee01b3bb8 P acc217e1270e4e56ac27f8bbe6ed20ba: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.003s
I20260812 06:57:50.518424 3676 tablet_bootstrap.cc:492] T ae36909a86394e4fada3c2cee01b3bb8 P 53890c7692834a24bd8919a9cbae000a: No bootstrap required, opened a new log
I20260812 06:57:50.518532 3676 ts_tablet_manager.cc:1397] T ae36909a86394e4fada3c2cee01b3bb8 P 53890c7692834a24bd8919a9cbae000a: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260812 06:57:50.518971 3676 raft_consensus.cc:348] T ae36909a86394e4fada3c2cee01b3bb8 P 53890c7692834a24bd8919a9cbae000a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c25a0ae5f88b4400a21eb4c9085cd2c3" member_type: VOTER last_known_addr { host: "127.19.52.67" port: 46061 } } peers { permanent_uuid: "53890c7692834a24bd8919a9cbae000a" member_type: VOTER last_known_addr { host: "127.19.52.65" port: 43193 } } peers { permanent_uuid: "acc217e1270e4e56ac27f8bbe6ed20ba" member_type: VOTER last_known_addr { host: "127.19.52.66" port: 36889 } }
I20260812 06:57:50.519093 3676 raft_consensus.cc:374] T ae36909a86394e4fada3c2cee01b3bb8 P 53890c7692834a24bd8919a9cbae000a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 06:57:50.519129 3676 raft_consensus.cc:729] T ae36909a86394e4fada3c2cee01b3bb8 P 53890c7692834a24bd8919a9cbae000a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 53890c7692834a24bd8919a9cbae000a, State: Initialized, Role: FOLLOWER
I20260812 06:57:50.518740 4040 tablet_service.cc:1467] Processing CreateTablet for tablet ae36909a86394e4fada3c2cee01b3bb8 (DEFAULT_TABLE table=test-table [id=36c00697b24c43e9b9ac20a6b3f4012e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 06:57:50.519263 3676 consensus_queue.cc:260] T ae36909a86394e4fada3c2cee01b3bb8 P 53890c7692834a24bd8919a9cbae000a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c25a0ae5f88b4400a21eb4c9085cd2c3" member_type: VOTER last_known_addr { host: "127.19.52.67" port: 46061 } } peers { permanent_uuid: "53890c7692834a24bd8919a9cbae000a" member_type: VOTER last_known_addr { host: "127.19.52.65" port: 43193 } } peers { permanent_uuid: "acc217e1270e4e56ac27f8bbe6ed20ba" member_type: VOTER last_known_addr { host: "127.19.52.66" port: 36889 } }
I20260812 06:57:50.519529 3676 ts_tablet_manager.cc:1428] T ae36909a86394e4fada3c2cee01b3bb8 P 53890c7692834a24bd8919a9cbae000a: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260812 06:57:50.519737 3837 raft_consensus.cc:348] T ae36909a86394e4fada3c2cee01b3bb8 P acc217e1270e4e56ac27f8bbe6ed20ba [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c25a0ae5f88b4400a21eb4c9085cd2c3" member_type: VOTER last_known_addr { host: "127.19.52.67" port: 46061 } } peers { permanent_uuid: "53890c7692834a24bd8919a9cbae000a" member_type: VOTER last_known_addr { host: "127.19.52.65" port: 43193 } } peers { permanent_uuid: "acc217e1270e4e56ac27f8bbe6ed20ba" member_type: VOTER last_known_addr { host: "127.19.52.66" port: 36889 } }
I20260812 06:57:50.519865 3837 raft_consensus.cc:374] T ae36909a86394e4fada3c2cee01b3bb8 P acc217e1270e4e56ac27f8bbe6ed20ba [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 06:57:50.519906 3837 raft_consensus.cc:729] T ae36909a86394e4fada3c2cee01b3bb8 P acc217e1270e4e56ac27f8bbe6ed20ba [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: acc217e1270e4e56ac27f8bbe6ed20ba, State: Initialized, Role: FOLLOWER
I20260812 06:57:50.519919 4040 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ae36909a86394e4fada3c2cee01b3bb8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 06:57:50.520035 3837 consensus_queue.cc:260] T ae36909a86394e4fada3c2cee01b3bb8 P acc217e1270e4e56ac27f8bbe6ed20ba [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c25a0ae5f88b4400a21eb4c9085cd2c3" member_type: VOTER last_known_addr { host: "127.19.52.67" port: 46061 } } peers { permanent_uuid: "53890c7692834a24bd8919a9cbae000a" member_type: VOTER last_known_addr { host: "127.19.52.65" port: 43193 } } peers { permanent_uuid: "acc217e1270e4e56ac27f8bbe6ed20ba" member_type: VOTER last_known_addr { host: "127.19.52.66" port: 36889 } }
I20260812 06:57:50.520757 3837 ts_tablet_manager.cc:1428] T ae36909a86394e4fada3c2cee01b3bb8 P acc217e1270e4e56ac27f8bbe6ed20ba: Time spent starting tablet: real 0.003s user 0.000s sys 0.001s
W20260812 06:57:50.522079 3801 tablet.cc:2369] T ae36909a86394e4fada3c2cee01b3bb8 P 53890c7692834a24bd8919a9cbae000a: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 06:57:50.528465 4130 tablet_bootstrap.cc:492] T ae36909a86394e4fada3c2cee01b3bb8 P c25a0ae5f88b4400a21eb4c9085cd2c3: Bootstrap starting.
I20260812 06:57:50.530480 4130 tablet_bootstrap.cc:654] T ae36909a86394e4fada3c2cee01b3bb8 P c25a0ae5f88b4400a21eb4c9085cd2c3: Neither blocks nor log segments found. Creating new log.
I20260812 06:57:50.531840 4130 log.cc:824] T ae36909a86394e4fada3c2cee01b3bb8 P c25a0ae5f88b4400a21eb4c9085cd2c3: Log is configured to *not* fsync() on all Append() calls
I20260812 06:57:50.533885 4130 tablet_bootstrap.cc:492] T ae36909a86394e4fada3c2cee01b3bb8 P c25a0ae5f88b4400a21eb4c9085cd2c3: No bootstrap required, opened a new log
I20260812 06:57:50.534123 4130 ts_tablet_manager.cc:1397] T ae36909a86394e4fada3c2cee01b3bb8 P c25a0ae5f88b4400a21eb4c9085cd2c3: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.004s
I20260812 06:57:50.537470 4130 raft_consensus.cc:348] T ae36909a86394e4fada3c2cee01b3bb8 P c25a0ae5f88b4400a21eb4c9085cd2c3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c25a0ae5f88b4400a21eb4c9085cd2c3" member_type: VOTER last_known_addr { host: "127.19.52.67" port: 46061 } } peers { permanent_uuid: "53890c7692834a24bd8919a9cbae000a" member_type: VOTER last_known_addr { host: "127.19.52.65" port: 43193 } } peers { permanent_uuid: "acc217e1270e4e56ac27f8bbe6ed20ba" member_type: VOTER last_known_addr { host: "127.19.52.66" port: 36889 } }
I20260812 06:57:50.537760 4130 raft_consensus.cc:374] T ae36909a86394e4fada3c2cee01b3bb8 P c25a0ae5f88b4400a21eb4c9085cd2c3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 06:57:50.537846 4130 raft_consensus.cc:729] T ae36909a86394e4fada3c2cee01b3bb8 P c25a0ae5f88b4400a21eb4c9085cd2c3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c25a0ae5f88b4400a21eb4c9085cd2c3, State: Initialized, Role: FOLLOWER
I20260812 06:57:50.538340 4130 consensus_queue.cc:260] T ae36909a86394e4fada3c2cee01b3bb8 P c25a0ae5f88b4400a21eb4c9085cd2c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c25a0ae5f88b4400a21eb4c9085cd2c3" member_type: VOTER last_known_addr { host: "127.19.52.67" port: 46061 } } peers { permanent_uuid: "53890c7692834a24bd8919a9cbae000a" member_type: VOTER last_known_addr { host: "127.19.52.65" port: 43193 } } peers { permanent_uuid: "acc217e1270e4e56ac27f8bbe6ed20ba" member_type: VOTER last_known_addr { host: "127.19.52.66" port: 36889 } }
I20260812 06:57:50.539088 4117 heartbeater.cc:503] Master 127.19.52.126:42629 was elected leader, sending a full tablet report...
I20260812 06:57:50.539443 4130 ts_tablet_manager.cc:1428] T ae36909a86394e4fada3c2cee01b3bb8 P c25a0ae5f88b4400a21eb4c9085cd2c3: Time spent starting tablet: real 0.005s user 0.005s sys 0.001s
I20260812 06:57:50.602221 3809 raft_consensus.cc:482] T ae36909a86394e4fada3c2cee01b3bb8 P 53890c7692834a24bd8919a9cbae000a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 06:57:50.602540 3809 raft_consensus.cc:504] T ae36909a86394e4fada3c2cee01b3bb8 P 53890c7692834a24bd8919a9cbae000a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c25a0ae5f88b4400a21eb4c9085cd2c3" member_type: VOTER last_known_addr { host: "127.19.52.67" port: 46061 } } peers { permanent_uuid: "53890c7692834a24bd8919a9cbae000a" member_type: VOTER last_known_addr { host: "127.19.52.65" port: 43193 } } peers { permanent_uuid: "acc217e1270e4e56ac27f8bbe6ed20ba" member_type: VOTER last_known_addr { host: "127.19.52.66" port: 36889 } }
I20260812 06:57:50.603713 3809 leader_election.cc:288] T ae36909a86394e4fada3c2cee01b3bb8 P 53890c7692834a24bd8919a9cbae000a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c25a0ae5f88b4400a21eb4c9085cd2c3 (127.19.52.67:46061), acc217e1270e4e56ac27f8bbe6ed20ba (127.19.52.66:36889)
I20260812 06:57:50.609350 4127 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:57:50.604228 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.52.65:37663 (local address 127.19.52.67:46061)
0812 06:57:50.604371 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0812 06:57:50.604376 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 06:57:50.604602 (+ 226us) server_negotiation.cc:407] Connection header received
0812 06:57:50.604774 (+ 172us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:57:50.604778 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:57:50.604833 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:57:50.604943 (+ 110us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 06:57:50.605397 (+ 454us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:57:50.606217 (+ 820us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:57:50.607967 (+ 1750us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:57:50.608648 (+ 681us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:57:50.608801 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:57:50.608868 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0812 06:57:50.609133 (+ 265us) server_negotiation.cc:299] Negotiation successful
0812 06:57:50.609208 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260812 06:57:50.609385 3971 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:57:50.604224 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.52.65:32833 (local address 127.19.52.66:36889)
0812 06:57:50.604371 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0812 06:57:50.604376 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 06:57:50.604599 (+ 223us) server_negotiation.cc:407] Connection header received
0812 06:57:50.604774 (+ 175us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:57:50.604778 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:57:50.604849 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:57:50.604946 (+ 97us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 06:57:50.605397 (+ 451us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:57:50.606221 (+ 824us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:57:50.607966 (+ 1745us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:57:50.608696 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:57:50.608814 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:57:50.608988 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0812 06:57:50.609132 (+ 144us) server_negotiation.cc:299] Negotiation successful
0812 06:57:50.609214 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260812 06:57:50.609390 4134 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:57:50.604010 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.66:36889 (local address 127.19.52.65:32833)
0812 06:57:50.604407 (+ 397us) negotiation.cc:107] Waiting for socket to connect
0812 06:57:50.604424 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0812 06:57:50.604621 (+ 197us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:57:50.604982 (+ 361us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:57:50.604985 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:57:50.605014 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 06:57:50.605265 (+ 251us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:57:50.605273 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:57:50.606370 (+ 1097us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:57:50.606374 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:57:50.607795 (+ 1421us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:57:50.607807 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:57:50.608816 (+ 1009us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:57:50.608818 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:57:50.608942 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:57:50.609003 (+ 61us) client_negotiation.cc:769] Sending connection context
0812 06:57:50.609091 (+ 88us) client_negotiation.cc:240] Negotiation successful
0812 06:57:50.609202 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":293,"mutex_wait_us":160,"spinlock_wait_cycles":32896,"thread_start_us":113,"threads_started":1}
I20260812 06:57:50.609396 4133 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:57:50.604010 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.67:46061 (local address 127.19.52.65:37663)
0812 06:57:50.604446 (+ 436us) negotiation.cc:107] Waiting for socket to connect
0812 06:57:50.604459 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 06:57:50.604621 (+ 162us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:57:50.604982 (+ 361us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:57:50.604985 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:57:50.605014 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 06:57:50.605265 (+ 251us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:57:50.605273 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:57:50.606370 (+ 1097us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:57:50.606374 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:57:50.607795 (+ 1421us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:57:50.607807 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:57:50.608810 (+ 1003us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:57:50.608814 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:57:50.608990 (+ 176us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:57:50.609007 (+ 17us) client_negotiation.cc:769] Sending connection context
0812 06:57:50.609091 (+ 84us) client_negotiation.cc:240] Negotiation successful
0812 06:57:50.609202 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":375,"mutex_wait_us":189,"spinlock_wait_cycles":35072,"thread_start_us":75,"threads_started":1}
I20260812 06:57:50.610183 3906 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ae36909a86394e4fada3c2cee01b3bb8" candidate_uuid: "53890c7692834a24bd8919a9cbae000a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "acc217e1270e4e56ac27f8bbe6ed20ba" is_pre_election: true
I20260812 06:57:50.610251 4063 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ae36909a86394e4fada3c2cee01b3bb8" candidate_uuid: "53890c7692834a24bd8919a9cbae000a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c25a0ae5f88b4400a21eb4c9085cd2c3" is_pre_election: true
I20260812 06:57:50.610559 4063 raft_consensus.cc:2454] T ae36909a86394e4fada3c2cee01b3bb8 P c25a0ae5f88b4400a21eb4c9085cd2c3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53890c7692834a24bd8919a9cbae000a in term 0.
I20260812 06:57:50.610522 3906 raft_consensus.cc:2454] T ae36909a86394e4fada3c2cee01b3bb8 P acc217e1270e4e56ac27f8bbe6ed20ba [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53890c7692834a24bd8919a9cbae000a in term 0.
I20260812 06:57:50.611083 3670 leader_election.cc:302] T ae36909a86394e4fada3c2cee01b3bb8 P 53890c7692834a24bd8919a9cbae000a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 53890c7692834a24bd8919a9cbae000a, acc217e1270e4e56ac27f8bbe6ed20ba, c25a0ae5f88b4400a21eb4c9085cd2c3; no voters:
I20260812 06:57:50.611388 3809 raft_consensus.cc:2781] T ae36909a86394e4fada3c2cee01b3bb8 P 53890c7692834a24bd8919a9cbae000a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 06:57:50.611481 3809 raft_consensus.cc:482] T ae36909a86394e4fada3c2cee01b3bb8 P 53890c7692834a24bd8919a9cbae000a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 06:57:50.611527 3809 raft_consensus.cc:3037] T ae36909a86394e4fada3c2cee01b3bb8 P 53890c7692834a24bd8919a9cbae000a [term 0 FOLLOWER]: Advancing to term 1
I20260812 06:57:50.612815 3809 raft_consensus.cc:504] T ae36909a86394e4fada3c2cee01b3bb8 P 53890c7692834a24bd8919a9cbae000a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c25a0ae5f88b4400a21eb4c9085cd2c3" member_type: VOTER last_known_addr { host: "127.19.52.67" port: 46061 } } peers { permanent_uuid: "53890c7692834a24bd8919a9cbae000a" member_type: VOTER last_known_addr { host: "127.19.52.65" port: 43193 } } peers { permanent_uuid: "acc217e1270e4e56ac27f8bbe6ed20ba" member_type: VOTER last_known_addr { host: "127.19.52.66" port: 36889 } }
I20260812 06:57:50.613209 3809 leader_election.cc:288] T ae36909a86394e4fada3c2cee01b3bb8 P 53890c7692834a24bd8919a9cbae000a [CANDIDATE]: Term 1 election: Requested vote from peers c25a0ae5f88b4400a21eb4c9085cd2c3 (127.19.52.67:46061), acc217e1270e4e56ac27f8bbe6ed20ba (127.19.52.66:36889)
I20260812 06:57:50.613593 3906 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ae36909a86394e4fada3c2cee01b3bb8" candidate_uuid: "53890c7692834a24bd8919a9cbae000a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "acc217e1270e4e56ac27f8bbe6ed20ba"
I20260812 06:57:50.613760 3906 raft_consensus.cc:3037] T ae36909a86394e4fada3c2cee01b3bb8 P acc217e1270e4e56ac27f8bbe6ed20ba [term 0 FOLLOWER]: Advancing to term 1
I20260812 06:57:50.613790 4063 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ae36909a86394e4fada3c2cee01b3bb8" candidate_uuid: "53890c7692834a24bd8919a9cbae000a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c25a0ae5f88b4400a21eb4c9085cd2c3"
I20260812 06:57:50.613933 4063 raft_consensus.cc:3037] T ae36909a86394e4fada3c2cee01b3bb8 P c25a0ae5f88b4400a21eb4c9085cd2c3 [term 0 FOLLOWER]: Advancing to term 1
I20260812 06:57:50.614905 3906 raft_consensus.cc:2454] T ae36909a86394e4fada3c2cee01b3bb8 P acc217e1270e4e56ac27f8bbe6ed20ba [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 53890c7692834a24bd8919a9cbae000a in term 1.
I20260812 06:57:50.615100 4063 raft_consensus.cc:2454] T ae36909a86394e4fada3c2cee01b3bb8 P c25a0ae5f88b4400a21eb4c9085cd2c3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 53890c7692834a24bd8919a9cbae000a in term 1.
I20260812 06:57:50.615299 3670 leader_election.cc:302] T ae36909a86394e4fada3c2cee01b3bb8 P 53890c7692834a24bd8919a9cbae000a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53890c7692834a24bd8919a9cbae000a, acc217e1270e4e56ac27f8bbe6ed20ba; no voters:
I20260812 06:57:50.615512 3809 raft_consensus.cc:2781] T ae36909a86394e4fada3c2cee01b3bb8 P 53890c7692834a24bd8919a9cbae000a [term 1 FOLLOWER]: Leader election won for term 1
I20260812 06:57:50.615751 3809 raft_consensus.cc:686] T ae36909a86394e4fada3c2cee01b3bb8 P 53890c7692834a24bd8919a9cbae000a [term 1 LEADER]: Becoming Leader. State: Replica: 53890c7692834a24bd8919a9cbae000a, State: Running, Role: LEADER
I20260812 06:57:50.616020 3809 consensus_queue.cc:237] T ae36909a86394e4fada3c2cee01b3bb8 P 53890c7692834a24bd8919a9cbae000a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c25a0ae5f88b4400a21eb4c9085cd2c3" member_type: VOTER last_known_addr { host: "127.19.52.67" port: 46061 } } peers { permanent_uuid: "53890c7692834a24bd8919a9cbae000a" member_type: VOTER last_known_addr { host: "127.19.52.65" port: 43193 } } peers { permanent_uuid: "acc217e1270e4e56ac27f8bbe6ed20ba" member_type: VOTER last_known_addr { host: "127.19.52.66" port: 36889 } }
I20260812 06:57:50.619222 3590 catalog_manager.cc:5495] T ae36909a86394e4fada3c2cee01b3bb8 P 53890c7692834a24bd8919a9cbae000a reported cstate change: term changed from 0 to 1, leader changed from <none> to 53890c7692834a24bd8919a9cbae000a (127.19.52.65). New cstate: current_term: 1 leader_uuid: "53890c7692834a24bd8919a9cbae000a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c25a0ae5f88b4400a21eb4c9085cd2c3" member_type: VOTER last_known_addr { host: "127.19.52.67" port: 46061 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "53890c7692834a24bd8919a9cbae000a" member_type: VOTER last_known_addr { host: "127.19.52.65" port: 43193 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "acc217e1270e4e56ac27f8bbe6ed20ba" member_type: VOTER last_known_addr { host: "127.19.52.66" port: 36889 } health_report { overall_health: UNKNOWN } } }
I20260812 06:57:50.652313 4139 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:57:50.648152 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51708 (local address 127.19.52.65:43193)
0812 06:57:50.648487 (+ 335us) server_negotiation.cc:206] Beginning negotiation
0812 06:57:50.648494 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 06:57:50.648509 (+ 15us) server_negotiation.cc:407] Connection header received
0812 06:57:50.648627 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:57:50.648629 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:57:50.648687 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:57:50.648794 (+ 107us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 06:57:50.649111 (+ 317us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:57:50.649688 (+ 577us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:57:50.650668 (+ 980us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:57:50.650906 (+ 238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:57:50.651039 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:57:50.651369 (+ 330us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 06:57:50.651879 (+ 510us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 06:57:50.651991 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0812 06:57:50.652107 (+ 116us) server_negotiation.cc:299] Negotiation successful
0812 06:57:50.652166 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":259,"thread_start_us":155,"threads_started":1}
I20260812 06:57:50.657021 3906 raft_consensus.cc:1261] T ae36909a86394e4fada3c2cee01b3bb8 P acc217e1270e4e56ac27f8bbe6ed20ba [term 1 FOLLOWER]: Refusing update from remote peer 53890c7692834a24bd8919a9cbae000a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 06:57:50.657054 4063 raft_consensus.cc:1261] T ae36909a86394e4fada3c2cee01b3bb8 P c25a0ae5f88b4400a21eb4c9085cd2c3 [term 1 FOLLOWER]: Refusing update from remote peer 53890c7692834a24bd8919a9cbae000a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 06:57:50.657886 3809 consensus_queue.cc:1035] T ae36909a86394e4fada3c2cee01b3bb8 P 53890c7692834a24bd8919a9cbae000a [LEADER]: Connected to new peer: Peer: permanent_uuid: "c25a0ae5f88b4400a21eb4c9085cd2c3" member_type: VOTER last_known_addr { host: "127.19.52.67" port: 46061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 06:57:50.658154 4136 consensus_queue.cc:1035] T ae36909a86394e4fada3c2cee01b3bb8 P 53890c7692834a24bd8919a9cbae000a [LEADER]: Connected to new peer: Peer: permanent_uuid: "acc217e1270e4e56ac27f8bbe6ed20ba" member_type: VOTER last_known_addr { host: "127.19.52.66" port: 36889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 06:57:50.663998 4140 mvcc.cc:205] Tried to move back new op lower bound from 7317577198203670528 to 7317577198046150656. Current Snapshot: MvccSnapshot[applied={T|T < 7317577198203670528}]
I20260812 06:57:50.664394 4141 mvcc.cc:205] Tried to move back new op lower bound from 7317577198203670528 to 7317577198046150656. Current Snapshot: MvccSnapshot[applied={T|T < 7317577198203670528}]
I20260812 06:57:50.675000 3590 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin'} at 127.0.0.1:44396:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 06:57:50.675230 3590 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin'} at 127.0.0.1:44396:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 06:57:50.677992 3590 catalog_manager.cc:5782] T 00000000000000000000000000000000 P cb9a783d342a46d4858bf6291d637d35: Sending DeleteTablet for 3 replicas of tablet ae36909a86394e4fada3c2cee01b3bb8
I20260812 06:57:50.678822 3882 tablet_service.cc:1514] Processing DeleteTablet for tablet ae36909a86394e4fada3c2cee01b3bb8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 06:57:50 UTC) from {username='kudu', principal='kudu/127.19.52.126@KRBTEST.COM'} at 127.0.0.1:51752
I20260812 06:57:50.678799 4040 tablet_service.cc:1514] Processing DeleteTablet for tablet ae36909a86394e4fada3c2cee01b3bb8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 06:57:50 UTC) from {username='kudu', principal='kudu/127.19.52.126@KRBTEST.COM'} at 127.0.0.1:44708
I20260812 06:57:50.678882 3727 tablet_service.cc:1514] Processing DeleteTablet for tablet ae36909a86394e4fada3c2cee01b3bb8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 06:57:50 UTC) from {username='kudu', principal='kudu/127.19.52.126@KRBTEST.COM'} at 127.0.0.1:51704
I20260812 06:57:50.679042 3973 tablet_replica.cc:330] T ae36909a86394e4fada3c2cee01b3bb8 P acc217e1270e4e56ac27f8bbe6ed20ba: stopping tablet replica
I20260812 06:57:50.679169 3973 raft_consensus.cc:2229] T ae36909a86394e4fada3c2cee01b3bb8 P acc217e1270e4e56ac27f8bbe6ed20ba [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 06:57:50.679319 3973 raft_consensus.cc:2258] T ae36909a86394e4fada3c2cee01b3bb8 P acc217e1270e4e56ac27f8bbe6ed20ba [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 06:57:50.679569 19665 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu with pid 3631
I20260812 06:57:50.679600 4151 tablet_replica.cc:330] T ae36909a86394e4fada3c2cee01b3bb8 P 53890c7692834a24bd8919a9cbae000a: stopping tablet replica
I20260812 06:57:50.679742 4151 raft_consensus.cc:2229] T ae36909a86394e4fada3c2cee01b3bb8 P 53890c7692834a24bd8919a9cbae000a [term 1 LEADER]: Raft consensus shutting down.
I20260812 06:57:50.680980 3973 ts_tablet_manager.cc:1905] T ae36909a86394e4fada3c2cee01b3bb8 P acc217e1270e4e56ac27f8bbe6ed20ba: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 06:57:50.684810 3973 ts_tablet_manager.cc:1918] T ae36909a86394e4fada3c2cee01b3bb8 P acc217e1270e4e56ac27f8bbe6ed20ba: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 06:57:50.684900 3973 log.cc:1196] T ae36909a86394e4fada3c2cee01b3bb8 P acc217e1270e4e56ac27f8bbe6ed20ba: Deleting WAL directory at /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517687384214-19665-0/minicluster-data/ts-1/wal/wals/ae36909a86394e4fada3c2cee01b3bb8
I20260812 06:57:50.685184 3973 ts_tablet_manager.cc:1939] T ae36909a86394e4fada3c2cee01b3bb8 P acc217e1270e4e56ac27f8bbe6ed20ba: Deleting consensus metadata
I20260812 06:57:50.685352 4152 tablet_replica.cc:330] T ae36909a86394e4fada3c2cee01b3bb8 P c25a0ae5f88b4400a21eb4c9085cd2c3: stopping tablet replica
I20260812 06:57:50.685734 4152 raft_consensus.cc:2229] T ae36909a86394e4fada3c2cee01b3bb8 P c25a0ae5f88b4400a21eb4c9085cd2c3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 06:57:50.686077 3572 catalog_manager.cc:4841] TS acc217e1270e4e56ac27f8bbe6ed20ba (127.19.52.66:36889): tablet ae36909a86394e4fada3c2cee01b3bb8 (table test-table [id=36c00697b24c43e9b9ac20a6b3f4012e]) successfully deleted
I20260812 06:57:50.686146 4152 raft_consensus.cc:2258] T ae36909a86394e4fada3c2cee01b3bb8 P c25a0ae5f88b4400a21eb4c9085cd2c3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 06:57:50.687542 4152 ts_tablet_manager.cc:1905] T ae36909a86394e4fada3c2cee01b3bb8 P c25a0ae5f88b4400a21eb4c9085cd2c3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 06:57:50.689884 4152 ts_tablet_manager.cc:1918] T ae36909a86394e4fada3c2cee01b3bb8 P c25a0ae5f88b4400a21eb4c9085cd2c3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 06:57:50.690030 4152 log.cc:1196] T ae36909a86394e4fada3c2cee01b3bb8 P c25a0ae5f88b4400a21eb4c9085cd2c3: Deleting WAL directory at /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517687384214-19665-0/minicluster-data/ts-2/wal/wals/ae36909a86394e4fada3c2cee01b3bb8
I20260812 06:57:50.690456 4152 ts_tablet_manager.cc:1939] T ae36909a86394e4fada3c2cee01b3bb8 P c25a0ae5f88b4400a21eb4c9085cd2c3: Deleting consensus metadata
W20260812 06:57:50.691280 3572 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.19.52.65:43193 (error 108)
I20260812 06:57:50.691447 19665 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu with pid 3804
I20260812 06:57:50.694916 3810 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:57:50.694510 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.65:43193 (local address 127.0.0.1:51716)
0812 06:57:50.694703 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0812 06:57:50.694798 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.52.65:43193: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":64}
I20260812 06:57:50.694970 3574 catalog_manager.cc:4841] TS c25a0ae5f88b4400a21eb4c9085cd2c3 (127.19.52.67:46061): tablet ae36909a86394e4fada3c2cee01b3bb8 (table test-table [id=36c00697b24c43e9b9ac20a6b3f4012e]) successfully deleted
W20260812 06:57:50.695128 3572 catalog_manager.cc:4584] TS 53890c7692834a24bd8919a9cbae000a (127.19.52.65:43193): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet ae36909a86394e4fada3c2cee01b3bb8: Network error: Client connection negotiation failed: client connection to 127.19.52.65:43193: connect: Connection refused (error 111)
I20260812 06:57:50.702039 19665 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu with pid 3970
I20260812 06:57:50.710306 19665 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu with pid 3555
I20260812 06:57:50.719697 19665 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu
/tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517687384214-19665-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517687384214-19665-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517687384214-19665-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517687384214-19665-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.52.126:42629
--webserver_interface=127.19.52.126
--webserver_port=42859
--keytab_file=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517687384214-19665-0/krb5kdc/kudu_127.19.52.126.keytab
--principal=kudu/127.19.52.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.52.84:33591
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.19.52.126:42629
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517687384214-19665-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517687384214-19665-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517687384214-19665-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 06:57:50.836630 4155 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:57:50.837042 4155 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:57:50.837155 4155 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 06:57:50.841477 4155 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 06:57:50.841595 4155 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 06:57:50.841638 4155 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 06:57:50.841673 4155 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 06:57:50.841701 4155 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 06:57:50.847424 4155 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.52.84:33591
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517687384214-19665-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517687384214-19665-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.19.52.126:42629
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517687384214-19665-0/krb5kdc/kudu_127.19.52.126.keytab
--principal=kudu/127.19.52.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.52.126:42629
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517687384214-19665-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.19.52.126
--webserver_port=42859
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4155
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517687384214-19665-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 06:15:54 UTC on ccycloud.quasar-rnfzce.root.comops.site
build id 4581989
I20260812 06:57:50.848728 4155 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:57:50.849858 4155 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 06:57:50.859038 4160 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:57:50.859237 4155 server_base.cc:1034] running on GCE node
W20260812 06:57:50.859025 4161 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:57:50.859025 4163 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:57:50.859977 4155 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:57:50.861217 4155 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:57:50.862403 4155 hybrid_clock.cc:648] HybridClock initialized: now 1786517870862392 us; error 32 us; skew 500 ppm
Aug 12 06:57:50 dist-test-slave-65mx krb5kdc[2923](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517870, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.52.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 06:57:50.865303 4155 init.cc:377] Logged in from keytab as kudu/127.19.52.126@KRBTEST.COM (short username kudu)
I20260812 06:57:50.866269 4155 webserver.cc:460] Webserver started at http://127.19.52.126:42859/ using document root <none> and password file <none>
I20260812 06:57:50.866923 4155 fs_manager.cc:359] Metadata directory not provided
I20260812 06:57:50.866993 4155 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:57:50.871037 4155 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260812 06:57:50.873624 4170 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 06:57:50.875056 4155 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260812 06:57:50.875233 4155 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517687384214-19665-0/minicluster-data/master-0/data,/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517687384214-19665-0/minicluster-data/master-0/wal
uuid: "cb9a783d342a46d4858bf6291d637d35"
format_stamp: "Formatted at 2026-08-12 06:57:48 on dist-test-slave-65mx"
server_key: "1165741c3419a9ea4d6292178791647f"
server_key_iv: "33a1a865a75e3f0cc43738e7bca493c4"
server_key_version: "encryptionkey@0"
I20260812 06:57:50.875852 4155 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517687384214-19665-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517687384214-19665-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517687384214-19665-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:57:50.897269 4155 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 06:57:50.900910 4155 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 06:57:50.901165 4155 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:57:50.909511 4155 rpc_server.cc:307] RPC server started. Bound to: 127.19.52.126:42629
I20260812 06:57:50.909830 4228 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.52.126:42629 every 8 connection(s)
I20260812 06:57:50.910668 4155 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517687384214-19665-0/minicluster-data/master-0/data/info.pb
I20260812 06:57:50.915153 4229 sys_catalog.cc:263] Verifying existing consensus state
I20260812 06:57:50.917001 4229 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cb9a783d342a46d4858bf6291d637d35: Bootstrap starting.
I20260812 06:57:50.918058 19665 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu as pid 4155
I20260812 06:57:50.918555 19665 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu
/tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517687384214-19665-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517687384214-19665-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517687384214-19665-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517687384214-19665-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.52.65:43193
--local_ip_for_outbound_sockets=127.19.52.65
--tserver_master_addrs=127.19.52.126:42629
--webserver_port=35641
--webserver_interface=127.19.52.65
--keytab_file=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517687384214-19665-0/krb5kdc/kudu_127.19.52.65.keytab
--principal=kudu/127.19.52.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.52.84:33591
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517687384214-19665-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517687384214-19665-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517687384214-19665-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 06:57:50.936364 4229 log.cc:824] T 00000000000000000000000000000000 P cb9a783d342a46d4858bf6291d637d35: Log is configured to *not* fsync() on all Append() calls
I20260812 06:57:50.948099 4229 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cb9a783d342a46d4858bf6291d637d35: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=7 ignored=0} mutations{seen=9 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260812 06:57:50.948623 4229 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cb9a783d342a46d4858bf6291d637d35: Bootstrap complete.
I20260812 06:57:50.952873 4229 raft_consensus.cc:348] T 00000000000000000000000000000000 P cb9a783d342a46d4858bf6291d637d35 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cb9a783d342a46d4858bf6291d637d35" member_type: VOTER last_known_addr { host: "127.19.52.126" port: 42629 } }
I20260812 06:57:50.953289 4229 raft_consensus.cc:729] T 00000000000000000000000000000000 P cb9a783d342a46d4858bf6291d637d35 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: cb9a783d342a46d4858bf6291d637d35, State: Initialized, Role: FOLLOWER
I20260812 06:57:50.953879 4229 consensus_queue.cc:260] T 00000000000000000000000000000000 P cb9a783d342a46d4858bf6291d637d35 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cb9a783d342a46d4858bf6291d637d35" member_type: VOTER last_known_addr { host: "127.19.52.126" port: 42629 } }
I20260812 06:57:50.954023 4229 raft_consensus.cc:388] T 00000000000000000000000000000000 P cb9a783d342a46d4858bf6291d637d35 [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 06:57:50.954116 4229 raft_consensus.cc:482] T 00000000000000000000000000000000 P cb9a783d342a46d4858bf6291d637d35 [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 06:57:50.954242 4229 raft_consensus.cc:3037] T 00000000000000000000000000000000 P cb9a783d342a46d4858bf6291d637d35 [term 2 FOLLOWER]: Advancing to term 3
I20260812 06:57:50.956275 4229 raft_consensus.cc:504] T 00000000000000000000000000000000 P cb9a783d342a46d4858bf6291d637d35 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cb9a783d342a46d4858bf6291d637d35" member_type: VOTER last_known_addr { host: "127.19.52.126" port: 42629 } }
I20260812 06:57:50.956665 4229 leader_election.cc:302] T 00000000000000000000000000000000 P cb9a783d342a46d4858bf6291d637d35 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: cb9a783d342a46d4858bf6291d637d35; no voters:
I20260812 06:57:50.957070 4229 leader_election.cc:288] T 00000000000000000000000000000000 P cb9a783d342a46d4858bf6291d637d35 [CANDIDATE]: Term 3 election: Requested vote from peers
I20260812 06:57:50.957158 4234 raft_consensus.cc:2781] T 00000000000000000000000000000000 P cb9a783d342a46d4858bf6291d637d35 [term 3 FOLLOWER]: Leader election won for term 3
I20260812 06:57:50.957340 4234 raft_consensus.cc:686] T 00000000000000000000000000000000 P cb9a783d342a46d4858bf6291d637d35 [term 3 LEADER]: Becoming Leader. State: Replica: cb9a783d342a46d4858bf6291d637d35, State: Running, Role: LEADER
I20260812 06:57:50.957647 4234 consensus_queue.cc:237] T 00000000000000000000000000000000 P cb9a783d342a46d4858bf6291d637d35 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cb9a783d342a46d4858bf6291d637d35" member_type: VOTER last_known_addr { host: "127.19.52.126" port: 42629 } }
I20260812 06:57:50.958376 4229 sys_catalog.cc:564] T 00000000000000000000000000000000 P cb9a783d342a46d4858bf6291d637d35 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 06:57:50.958678 4236 sys_catalog.cc:455] T 00000000000000000000000000000000 P cb9a783d342a46d4858bf6291d637d35 [sys.catalog]: SysCatalogTable state changed. Reason: New leader cb9a783d342a46d4858bf6291d637d35. Latest consensus state: current_term: 3 leader_uuid: "cb9a783d342a46d4858bf6291d637d35" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cb9a783d342a46d4858bf6291d637d35" member_type: VOTER last_known_addr { host: "127.19.52.126" port: 42629 } } }
I20260812 06:57:50.958811 4236 sys_catalog.cc:458] T 00000000000000000000000000000000 P cb9a783d342a46d4858bf6291d637d35 [sys.catalog]: This master's current role is: LEADER
I20260812 06:57:50.959044 4237 sys_catalog.cc:455] T 00000000000000000000000000000000 P cb9a783d342a46d4858bf6291d637d35 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "cb9a783d342a46d4858bf6291d637d35" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cb9a783d342a46d4858bf6291d637d35" member_type: VOTER last_known_addr { host: "127.19.52.126" port: 42629 } } }
I20260812 06:57:50.959154 4237 sys_catalog.cc:458] T 00000000000000000000000000000000 P cb9a783d342a46d4858bf6291d637d35 [sys.catalog]: This master's current role is: LEADER
I20260812 06:57:50.959219 4239 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 06:57:50.964581 4239 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 06:57:50.965262 4239 catalog_manager.cc:1259] Loaded cluster ID: e36be11f1aff48c080e5d4150585e637
I20260812 06:57:50.965325 4239 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
F20260812 06:57:50.966456 4239 catalog_manager.cc:1466] Initializing Kudu internal certificate authority failed: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7f29b077c9ad google::LogMessage::Fail() at ??:0
@ 0x7f29b077f88a google::LogMessage::SendToLog() at ??:0
@ 0x7f29b077c4de google::LogMessage::Flush() at ??:0
@ 0x7f29b077da19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7f29bb0bbfdd _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlSt8functionIFNS_6StatusEvEERKNS_9consensus13RaftConsensusElPKcE_clES5_S9_lSB_ at ??:0
@ 0x7f29bb0bcd2f kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7f29bb0b9579 _ZZN4kudu6master14CatalogManager17ElectedAsLeaderCbEvENKUlvE_clEv at ??:0
@ 0x7f29bb0fd640 _ZNSt17_Function_handlerIFvvEZN4kudu6master14CatalogManager17ElectedAsLeaderCbEvEUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x61d838 std::function<>::operator()() at ??:0
@ 0x7f29b29ce3d7 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7f29b29cec83 _ZZN4kudu10ThreadPool12CreateThreadEvENKUlvE_clEv at ??:0
@ 0x7f29b29d03f2 _ZNSt17_Function_handlerIFvvEZN4kudu10ThreadPool12CreateThreadEvEUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x61d838 std::function<>::operator()() at ??:0
@ 0x7f29b29bf512 kudu::Thread::SuperviseThread() at ??:0
@ 0x7f29b40036db start_thread at ??:0
@ 0x7f29af27a61f clone at ??:0
W20260812 06:57:51.052690 4232 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:57:51.053040 4232 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:57:51.053092 4232 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 06:57:51.057920 4232 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 06:57:51.057984 4232 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 06:57:51.058056 4232 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.52.65
I20260812 06:57:51.064172 4232 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.52.84:33591
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517687384214-19665-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517687384214-19665-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517687384214-19665-0/krb5kdc/kudu_127.19.52.65.keytab
--principal=kudu/127.19.52.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.52.65:43193
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517687384214-19665-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.19.52.65
--webserver_port=35641
--tserver_master_addrs=127.19.52.126:42629
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4232
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.52.65
--log_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517687384214-19665-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 06:15:54 UTC on ccycloud.quasar-rnfzce.root.comops.site
build id 4581989
I20260812 06:57:51.065426 4232 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:57:51.066655 4232 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 06:57:51.077792 4274 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:57:51.077869 4277 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:57:51.078276 4232 server_base.cc:1034] running on GCE node
W20260812 06:57:51.078313 4275 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:57:51.078869 4232 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:57:51.079727 4232 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:57:51.080928 4232 hybrid_clock.cc:648] HybridClock initialized: now 1786517871080900 us; error 76 us; skew 500 ppm
Aug 12 06:57:51 dist-test-slave-65mx krb5kdc[2923](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517871, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.52.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 06:57:51.084923 4232 init.cc:377] Logged in from keytab as kudu/127.19.52.65@KRBTEST.COM (short username kudu)
I20260812 06:57:51.086156 4232 webserver.cc:460] Webserver started at http://127.19.52.65:35641/ using document root <none> and password file <none>
I20260812 06:57:51.086898 4232 fs_manager.cc:359] Metadata directory not provided
I20260812 06:57:51.086998 4232 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:57:51.092267 4232 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.000s
I20260812 06:57:51.095069 4285 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 06:57:51.096771 4232 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260812 06:57:51.096907 4232 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517687384214-19665-0/minicluster-data/ts-0/data,/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517687384214-19665-0/minicluster-data/ts-0/wal
uuid: "53890c7692834a24bd8919a9cbae000a"
format_stamp: "Formatted at 2026-08-12 06:57:48 on dist-test-slave-65mx"
server_key: "7ff113d789f532f2b56580096c1e3507"
server_key_iv: "4b60a721ba9c2d1bca1c96839ab5cf2e"
server_key_version: "encryptionkey@0"
I20260812 06:57:51.097681 4232 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517687384214-19665-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517687384214-19665-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517687384214-19665-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:57:51.121619 4232 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 06:57:51.125350 4232 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 06:57:51.125592 4232 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:57:51.126268 4232 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 06:57:51.127488 4293 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260812 06:57:51.133191 4232 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260812 06:57:51.133316 4232 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.006s user 0.000s sys 0.000s
I20260812 06:57:51.133404 4232 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260812 06:57:51.135376 4232 ts_tablet_manager.cc:610] Registered 1 tablets
I20260812 06:57:51.135454 4232 ts_tablet_manager.cc:589] Time spent register tablets: real 0.002s user 0.000s sys 0.000s
I20260812 06:57:51.135739 4293 tablet_bootstrap.cc:492] T ae36909a86394e4fada3c2cee01b3bb8 P 53890c7692834a24bd8919a9cbae000a: Bootstrap starting.
I20260812 06:57:51.146845 4232 rpc_server.cc:307] RPC server started. Bound to: 127.19.52.65:43193
I20260812 06:57:51.147117 4418 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.52.65:43193 every 8 connection(s)
I20260812 06:57:51.148118 4232 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517687384214-19665-0/minicluster-data/ts-0/data/info.pb
I20260812 06:57:51.157300 19665 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu as pid 4232
I20260812 06:57:51.157845 19665 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu
/tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517687384214-19665-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517687384214-19665-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517687384214-19665-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517687384214-19665-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.52.66:36889
--local_ip_for_outbound_sockets=127.19.52.66
--tserver_master_addrs=127.19.52.126:42629
--webserver_port=34873
--webserver_interface=127.19.52.66
--keytab_file=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517687384214-19665-0/krb5kdc/kudu_127.19.52.66.keytab
--principal=kudu/127.19.52.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.52.84:33591
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517687384214-19665-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517687384214-19665-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517687384214-19665-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 06:57:51.159255 4293 log.cc:824] T ae36909a86394e4fada3c2cee01b3bb8 P 53890c7692834a24bd8919a9cbae000a: Log is configured to *not* fsync() on all Append() calls
I20260812 06:57:51.167492 4293 tablet_bootstrap.cc:492] T ae36909a86394e4fada3c2cee01b3bb8 P 53890c7692834a24bd8919a9cbae000a: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260812 06:57:51.167940 4293 tablet_bootstrap.cc:492] T ae36909a86394e4fada3c2cee01b3bb8 P 53890c7692834a24bd8919a9cbae000a: Bootstrap complete.
I20260812 06:57:51.169332 4293 ts_tablet_manager.cc:1397] T ae36909a86394e4fada3c2cee01b3bb8 P 53890c7692834a24bd8919a9cbae000a: Time spent bootstrapping tablet: real 0.034s user 0.018s sys 0.014s
I20260812 06:57:51.171939 4293 raft_consensus.cc:348] T ae36909a86394e4fada3c2cee01b3bb8 P 53890c7692834a24bd8919a9cbae000a [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c25a0ae5f88b4400a21eb4c9085cd2c3" member_type: VOTER last_known_addr { host: "127.19.52.67" port: 46061 } } peers { permanent_uuid: "53890c7692834a24bd8919a9cbae000a" member_type: VOTER last_known_addr { host: "127.19.52.65" port: 43193 } } peers { permanent_uuid: "acc217e1270e4e56ac27f8bbe6ed20ba" member_type: VOTER last_known_addr { host: "127.19.52.66" port: 36889 } }
I20260812 06:57:51.172412 4293 raft_consensus.cc:729] T ae36909a86394e4fada3c2cee01b3bb8 P 53890c7692834a24bd8919a9cbae000a [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 53890c7692834a24bd8919a9cbae000a, State: Initialized, Role: FOLLOWER
I20260812 06:57:51.173024 4293 consensus_queue.cc:260] T ae36909a86394e4fada3c2cee01b3bb8 P 53890c7692834a24bd8919a9cbae000a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c25a0ae5f88b4400a21eb4c9085cd2c3" member_type: VOTER last_known_addr { host: "127.19.52.67" port: 46061 } } peers { permanent_uuid: "53890c7692834a24bd8919a9cbae000a" member_type: VOTER last_known_addr { host: "127.19.52.65" port: 43193 } } peers { permanent_uuid: "acc217e1270e4e56ac27f8bbe6ed20ba" member_type: VOTER last_known_addr { host: "127.19.52.66" port: 36889 } }
I20260812 06:57:51.174100 4293 ts_tablet_manager.cc:1428] T ae36909a86394e4fada3c2cee01b3bb8 P 53890c7692834a24bd8919a9cbae000a: Time spent starting tablet: real 0.005s user 0.004s sys 0.000s
W20260812 06:57:51.235078 4422 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 06:57:51.150394 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.126:42629 (local address 127.19.52.65:49945)
0812 06:57:51.150917 (+ 523us) negotiation.cc:107] Waiting for socket to connect
0812 06:57:51.150987 (+ 70us) client_negotiation.cc:174] Beginning negotiation
0812 06:57:51.151815 (+ 828us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:57:51.234149 (+ 82334us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.52.126:42629: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":310,"thread_start_us":171,"threads_started":1}
W20260812 06:57:51.235780 4420 heartbeater.cc:644] Failed to heartbeat to 127.19.52.126:42629 (0 consecutive failures): Network error: Failed to ping master at 127.19.52.126:42629: Client connection negotiation failed: client connection to 127.19.52.126:42629: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260812 06:57:51.236672 4422 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:57:51.236301 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.126:42629 (local address 127.19.52.65:55671)
0812 06:57:51.236447 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0812 06:57:51.236597 (+ 150us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.52.126:42629: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":64}
I20260812 06:57:51.237465 4422 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:57:51.237222 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.126:42629 (local address 127.19.52.65:51137)
0812 06:57:51.237351 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0812 06:57:51.237407 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.52.126:42629: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
W20260812 06:57:51.237660 4420 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260812 06:57:51.294792 4424 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:57:51.295089 4424 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:57:51.295135 4424 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 06:57:51.299484 4424 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 06:57:51.299557 4424 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 06:57:51.299621 4424 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.52.66
I20260812 06:57:51.305377 4424 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.52.84:33591
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517687384214-19665-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517687384214-19665-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517687384214-19665-0/krb5kdc/kudu_127.19.52.66.keytab
--principal=kudu/127.19.52.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.52.66:36889
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517687384214-19665-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.19.52.66
--webserver_port=34873
--tserver_master_addrs=127.19.52.126:42629
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4424
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.52.66
--log_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517687384214-19665-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 06:15:54 UTC on ccycloud.quasar-rnfzce.root.comops.site
build id 4581989
I20260812 06:57:51.306591 4424 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:57:51.307607 4424 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 06:57:51.318044 4432 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:57:51.318044 4434 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:57:51.318044 4431 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:57:51.318356 4424 server_base.cc:1034] running on GCE node
I20260812 06:57:51.318763 4424 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:57:51.319428 4424 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:57:51.320636 4424 hybrid_clock.cc:648] HybridClock initialized: now 1786517871320622 us; error 25 us; skew 500 ppm
Aug 12 06:57:51 dist-test-slave-65mx krb5kdc[2923](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517871, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.52.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 06:57:51.323444 4424 init.cc:377] Logged in from keytab as kudu/127.19.52.66@KRBTEST.COM (short username kudu)
I20260812 06:57:51.324396 4424 webserver.cc:460] Webserver started at http://127.19.52.66:34873/ using document root <none> and password file <none>
I20260812 06:57:51.324999 4424 fs_manager.cc:359] Metadata directory not provided
I20260812 06:57:51.325066 4424 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:57:51.329049 4424 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260812 06:57:51.331357 4441 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 06:57:51.332829 4424 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 06:57:51.332969 4424 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517687384214-19665-0/minicluster-data/ts-1/data,/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517687384214-19665-0/minicluster-data/ts-1/wal
uuid: "acc217e1270e4e56ac27f8bbe6ed20ba"
format_stamp: "Formatted at 2026-08-12 06:57:48 on dist-test-slave-65mx"
server_key: "1dd0239db6827471cbbe582197123db3"
server_key_iv: "2427b2eef86a471923318659774f55fb"
server_key_version: "encryptionkey@0"
I20260812 06:57:51.333563 4424 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517687384214-19665-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517687384214-19665-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517687384214-19665-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:57:51.353874 4424 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 06:57:51.357714 4424 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 06:57:51.357973 4424 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:57:51.358656 4424 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 06:57:51.359606 4424 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 06:57:51.359699 4424 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 06:57:51.359771 4424 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 06:57:51.359823 4424 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 06:57:51.369529 4424 rpc_server.cc:307] RPC server started. Bound to: 127.19.52.66:36889
I20260812 06:57:51.369608 4568 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.52.66:36889 every 8 connection(s)
I20260812 06:57:51.370651 4424 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517687384214-19665-0/minicluster-data/ts-1/data/info.pb
I20260812 06:57:51.375133 4572 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:57:51.373172 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.126:42629 (local address 127.19.52.66:51909)
0812 06:57:51.373690 (+ 518us) negotiation.cc:107] Waiting for socket to connect
0812 06:57:51.373816 (+ 126us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.52.126:42629: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":308,"thread_start_us":130,"threads_started":1}
W20260812 06:57:51.375784 4569 heartbeater.cc:644] Failed to heartbeat to 127.19.52.126:42629 (0 consecutive failures): Network error: Failed to ping master at 127.19.52.126:42629: Client connection negotiation failed: client connection to 127.19.52.126:42629: connect: Connection refused (error 111)
I20260812 06:57:51.376482 4572 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:57:51.376247 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.126:42629 (local address 127.19.52.66:44585)
0812 06:57:51.376371 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0812 06:57:51.376423 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.52.126:42629: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
I20260812 06:57:51.377259 4572 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:57:51.377008 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.126:42629 (local address 127.19.52.66:46523)
0812 06:57:51.377162 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0812 06:57:51.377208 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.52.126:42629: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":80}
W20260812 06:57:51.377476 4569 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260812 06:57:51.377539 19665 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu as pid 4424
I20260812 06:57:51.378011 19665 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu
/tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517687384214-19665-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517687384214-19665-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517687384214-19665-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517687384214-19665-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.52.67:46061
--local_ip_for_outbound_sockets=127.19.52.67
--tserver_master_addrs=127.19.52.126:42629
--webserver_port=42985
--webserver_interface=127.19.52.67
--keytab_file=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517687384214-19665-0/krb5kdc/kudu_127.19.52.67.keytab
--principal=kudu/127.19.52.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.52.84:33591
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517687384214-19665-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517687384214-19665-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517687384214-19665-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 06:57:51.519474 4573 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:57:51.519812 4573 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:57:51.519903 4573 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 06:57:51.525445 4573 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 06:57:51.525520 4573 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 06:57:51.525624 4573 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.52.67
I20260812 06:57:51.531451 4573 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.52.84:33591
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517687384214-19665-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517687384214-19665-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517687384214-19665-0/krb5kdc/kudu_127.19.52.67.keytab
--principal=kudu/127.19.52.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.52.67:46061
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517687384214-19665-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.19.52.67
--webserver_port=42985
--tserver_master_addrs=127.19.52.126:42629
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4573
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.52.67
--log_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517687384214-19665-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 06:15:54 UTC on ccycloud.quasar-rnfzce.root.comops.site
build id 4581989
I20260812 06:57:51.532703 4573 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:57:51.533773 4573 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 06:57:51.544423 4580 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:57:51.544584 4573 server_base.cc:1034] running on GCE node
W20260812 06:57:51.544423 4582 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:57:51.544430 4579 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:57:51.545233 4573 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:57:51.545969 4573 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:57:51.547179 4573 hybrid_clock.cc:648] HybridClock initialized: now 1786517871547143 us; error 46 us; skew 500 ppm
Aug 12 06:57:51 dist-test-slave-65mx krb5kdc[2923](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517871, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.52.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 06:57:51.550565 4573 init.cc:377] Logged in from keytab as kudu/127.19.52.67@KRBTEST.COM (short username kudu)
I20260812 06:57:51.551698 4573 webserver.cc:460] Webserver started at http://127.19.52.67:42985/ using document root <none> and password file <none>
I20260812 06:57:51.552320 4573 fs_manager.cc:359] Metadata directory not provided
I20260812 06:57:51.552433 4573 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:57:51.556746 4573 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.000s
I20260812 06:57:51.559137 4592 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 06:57:51.560632 4573 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 06:57:51.560787 4573 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517687384214-19665-0/minicluster-data/ts-2/data,/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517687384214-19665-0/minicluster-data/ts-2/wal
uuid: "c25a0ae5f88b4400a21eb4c9085cd2c3"
format_stamp: "Formatted at 2026-08-12 06:57:49 on dist-test-slave-65mx"
server_key: "87c1d3b0e65710d14c7e34951a9c243f"
server_key_iv: "1df5e0e999d968e6721cdfc759ff71bc"
server_key_version: "encryptionkey@0"
I20260812 06:57:51.561443 4573 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517687384214-19665-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517687384214-19665-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517687384214-19665-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:57:51.578805 4573 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 06:57:51.582358 4573 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 06:57:51.582597 4573 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:57:51.583287 4573 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 06:57:51.584189 4573 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 06:57:51.584270 4573 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 06:57:51.584340 4573 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 06:57:51.584395 4573 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 06:57:51.595728 4573 rpc_server.cc:307] RPC server started. Bound to: 127.19.52.67:46061
I20260812 06:57:51.595784 4725 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.52.67:46061 every 8 connection(s)
I20260812 06:57:51.596900 4573 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517687384214-19665-0/minicluster-data/ts-2/data/info.pb
I20260812 06:57:51.597914 19665 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu as pid 4573
I20260812 06:57:51.602388 4731 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:57:51.600327 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.126:42629 (local address 127.19.52.67:35293)
0812 06:57:51.600992 (+ 665us) negotiation.cc:107] Waiting for socket to connect
0812 06:57:51.601116 (+ 124us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.52.126:42629: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":407,"thread_start_us":211,"threads_started":1}
W20260812 06:57:51.603024 4727 heartbeater.cc:644] Failed to heartbeat to 127.19.52.126:42629 (0 consecutive failures): Network error: Failed to ping master at 127.19.52.126:42629: Client connection negotiation failed: client connection to 127.19.52.126:42629: connect: Connection refused (error 111)
I20260812 06:57:51.603749 4731 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:57:51.603461 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.126:42629 (local address 127.19.52.67:44363)
0812 06:57:51.603588 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0812 06:57:51.603670 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.52.126:42629: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
I20260812 06:57:51.604476 4731 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:57:51.604252 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.126:42629 (local address 127.19.52.67:55077)
0812 06:57:51.604360 (+ 108us) negotiation.cc:107] Waiting for socket to connect
0812 06:57:51.604404 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.52.126:42629: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260812 06:57:51.604679 4727 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260812 06:57:52.238982 4733 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:57:52.238501 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.126:42629 (local address 127.19.52.65:56087)
0812 06:57:52.238800 (+ 299us) negotiation.cc:107] Waiting for socket to connect
0812 06:57:52.238865 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.52.126:42629: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":205,"thread_start_us":79,"threads_started":1}
I20260812 06:57:52.378722 4735 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:57:52.378204 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.126:42629 (local address 127.19.52.66:46803)
0812 06:57:52.378521 (+ 317us) negotiation.cc:107] Waiting for socket to connect
0812 06:57:52.378601 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.52.126:42629: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":226,"thread_start_us":78,"threads_started":1}
I20260812 06:57:52.598141 4736 raft_consensus.cc:482] T ae36909a86394e4fada3c2cee01b3bb8 P 53890c7692834a24bd8919a9cbae000a [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 06:57:52.598345 4736 raft_consensus.cc:504] T ae36909a86394e4fada3c2cee01b3bb8 P 53890c7692834a24bd8919a9cbae000a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c25a0ae5f88b4400a21eb4c9085cd2c3" member_type: VOTER last_known_addr { host: "127.19.52.67" port: 46061 } } peers { permanent_uuid: "53890c7692834a24bd8919a9cbae000a" member_type: VOTER last_known_addr { host: "127.19.52.65" port: 43193 } } peers { permanent_uuid: "acc217e1270e4e56ac27f8bbe6ed20ba" member_type: VOTER last_known_addr { host: "127.19.52.66" port: 36889 } }
I20260812 06:57:52.599888 4736 leader_election.cc:288] T ae36909a86394e4fada3c2cee01b3bb8 P 53890c7692834a24bd8919a9cbae000a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers c25a0ae5f88b4400a21eb4c9085cd2c3 (127.19.52.67:46061), acc217e1270e4e56ac27f8bbe6ed20ba (127.19.52.66:36889)
I20260812 06:57:52.605726 4740 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:57:52.605223 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.126:42629 (local address 127.19.52.67:48837)
0812 06:57:52.605508 (+ 285us) negotiation.cc:107] Waiting for socket to connect
0812 06:57:52.605587 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.52.126:42629: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":191,"thread_start_us":88,"threads_started":1}
Aug 12 06:57:52 dist-test-slave-65mx krb5kdc[2923](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517871, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.52.65@KRBTEST.COM for kudu/127.19.52.67@KRBTEST.COM
Aug 12 06:57:52 dist-test-slave-65mx krb5kdc[2923](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517871, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.52.65@KRBTEST.COM for kudu/127.19.52.66@KRBTEST.COM
I20260812 06:57:52.627003 4738 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:57:52.600417 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.66:36889 (local address 127.19.52.65:35467)
0812 06:57:52.601026 (+ 609us) negotiation.cc:107] Waiting for socket to connect
0812 06:57:52.601039 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 06:57:52.601147 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:57:52.603205 (+ 2058us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:57:52.603208 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:57:52.603446 (+ 238us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:57:52.603946 (+ 500us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:57:52.603965 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:57:52.605599 (+ 1634us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:57:52.605606 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:57:52.606676 (+ 1070us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:57:52.606687 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:57:52.607448 (+ 761us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:57:52.607451 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:57:52.607615 (+ 164us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:57:52.618113 (+ 10498us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 06:57:52.618150 (+ 37us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:57:52.620606 (+ 2456us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 06:57:52.623533 (+ 2927us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:57:52.623541 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:57:52.623557 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:57:52.624068 (+ 511us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:57:52.624618 (+ 550us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:57:52.624624 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:57:52.624626 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:57:52.624876 (+ 250us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:57:52.625787 (+ 911us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 06:57:52.625794 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 06:57:52.626132 (+ 338us) client_negotiation.cc:769] Sending connection context
0812 06:57:52.626367 (+ 235us) client_negotiation.cc:240] Negotiation successful
0812 06:57:52.626789 (+ 422us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":547,"mutex_wait_us":116,"spinlock_wait_cycles":18816,"thread_start_us":76,"threads_started":1}
I20260812 06:57:52.627004 4733 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:57:52.600090 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.67:46061 (local address 127.19.52.65:46137)
0812 06:57:52.600251 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0812 06:57:52.600266 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 06:57:52.600391 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:57:52.602821 (+ 2430us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:57:52.602828 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:57:52.603323 (+ 495us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:57:52.603948 (+ 625us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:57:52.603997 (+ 49us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:57:52.605666 (+ 1669us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:57:52.605669 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:57:52.606680 (+ 1011us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:57:52.606690 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:57:52.608451 (+ 1761us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:57:52.608454 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:57:52.608518 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:57:52.618095 (+ 9577us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 06:57:52.618221 (+ 126us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:57:52.620204 (+ 1983us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 06:57:52.623537 (+ 3333us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:57:52.623541 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:57:52.623557 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:57:52.624068 (+ 511us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:57:52.624635 (+ 567us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:57:52.624638 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:57:52.624640 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:57:52.624876 (+ 236us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:57:52.625787 (+ 911us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 06:57:52.625794 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 06:57:52.626132 (+ 338us) client_negotiation.cc:769] Sending connection context
0812 06:57:52.626367 (+ 235us) client_negotiation.cc:240] Negotiation successful
0812 06:57:52.626841 (+ 474us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":61,"mutex_wait_us":456,"spinlock_wait_cycles":39168}
I20260812 06:57:52.627085 4737 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:57:52.600308 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.52.65:46137 (local address 127.19.52.67:46061)
0812 06:57:52.600945 (+ 637us) server_negotiation.cc:206] Beginning negotiation
0812 06:57:52.600959 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0812 06:57:52.601003 (+ 44us) server_negotiation.cc:407] Connection header received
0812 06:57:52.601934 (+ 931us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:57:52.601943 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:57:52.602388 (+ 445us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:57:52.602724 (+ 336us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:57:52.604168 (+ 1444us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:57:52.605477 (+ 1309us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:57:52.607688 (+ 2211us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:57:52.608326 (+ 638us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:57:52.608443 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:57:52.620618 (+ 12175us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:57:52.620628 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:57:52.620651 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:57:52.620705 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:57:52.623353 (+ 2648us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:57:52.624249 (+ 896us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:57:52.624256 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:57:52.624261 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:57:52.624445 (+ 184us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:57:52.625015 (+ 570us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:57:52.625018 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:57:52.625020 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:57:52.625649 (+ 629us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:57:52.625789 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0812 06:57:52.626565 (+ 776us) server_negotiation.cc:299] Negotiation successful
0812 06:57:52.626882 (+ 317us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":479,"thread_start_us":115,"threads_started":1}
I20260812 06:57:52.627681 4739 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:57:52.600679 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.52.65:35467 (local address 127.19.52.66:36889)
0812 06:57:52.601006 (+ 327us) server_negotiation.cc:206] Beginning negotiation
0812 06:57:52.601046 (+ 40us) server_negotiation.cc:399] Waiting for connection header
0812 06:57:52.601119 (+ 73us) server_negotiation.cc:407] Connection header received
0812 06:57:52.601934 (+ 815us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:57:52.601943 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:57:52.602388 (+ 445us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:57:52.602724 (+ 336us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:57:52.604186 (+ 1462us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:57:52.605412 (+ 1226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:57:52.606826 (+ 1414us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:57:52.607307 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:57:52.607509 (+ 202us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:57:52.620869 (+ 13360us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:57:52.620874 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:57:52.620884 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:57:52.620919 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:57:52.623352 (+ 2433us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:57:52.624249 (+ 897us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:57:52.624256 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:57:52.624261 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:57:52.624473 (+ 212us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:57:52.625015 (+ 542us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:57:52.625018 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:57:52.625020 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:57:52.625648 (+ 628us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:57:52.625789 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0812 06:57:52.626579 (+ 790us) server_negotiation.cc:299] Negotiation successful
0812 06:57:52.626882 (+ 303us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":221,"thread_start_us":100,"threads_started":1}
W20260812 06:57:52.629944 4286 leader_election.cc:341] T ae36909a86394e4fada3c2cee01b3bb8 P 53890c7692834a24bd8919a9cbae000a [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer acc217e1270e4e56ac27f8bbe6ed20ba (127.19.52.66:36889): Not found: Tablet not found: ae36909a86394e4fada3c2cee01b3bb8
W20260812 06:57:52.630191 4289 leader_election.cc:341] T ae36909a86394e4fada3c2cee01b3bb8 P 53890c7692834a24bd8919a9cbae000a [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer c25a0ae5f88b4400a21eb4c9085cd2c3 (127.19.52.67:46061): Not found: Tablet not found: ae36909a86394e4fada3c2cee01b3bb8
I20260812 06:57:52.630298 4289 leader_election.cc:302] T ae36909a86394e4fada3c2cee01b3bb8 P 53890c7692834a24bd8919a9cbae000a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53890c7692834a24bd8919a9cbae000a; no voters: acc217e1270e4e56ac27f8bbe6ed20ba, c25a0ae5f88b4400a21eb4c9085cd2c3
I20260812 06:57:52.630641 4736 raft_consensus.cc:2726] T ae36909a86394e4fada3c2cee01b3bb8 P 53890c7692834a24bd8919a9cbae000a [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260812 06:57:53.240320 4741 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:57:53.239883 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.126:42629 (local address 127.19.52.65:53005)
0812 06:57:53.240161 (+ 278us) negotiation.cc:107] Waiting for socket to connect
0812 06:57:53.240224 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.52.126:42629: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":193,"thread_start_us":112,"threads_started":1}
I20260812 06:57:53.380165 4742 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:57:53.379616 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.126:42629 (local address 127.19.52.66:45365)
0812 06:57:53.379967 (+ 351us) negotiation.cc:107] Waiting for socket to connect
0812 06:57:53.380065 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.52.126:42629: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":279,"thread_start_us":115,"threads_started":1}
I20260812 06:57:53.607059 4743 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:57:53.606599 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.126:42629 (local address 127.19.52.67:44919)
0812 06:57:53.606887 (+ 288us) negotiation.cc:107] Waiting for socket to connect
0812 06:57:53.606959 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.52.126:42629: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":203,"thread_start_us":84,"threads_started":1}
I20260812 06:57:54.241694 4744 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:57:54.241230 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.126:42629 (local address 127.19.52.65:60787)
0812 06:57:54.241531 (+ 301us) negotiation.cc:107] Waiting for socket to connect
0812 06:57:54.241592 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.52.126:42629: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":220,"thread_start_us":82,"threads_started":1}
I20260812 06:57:54.381721 4745 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:57:54.381157 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.126:42629 (local address 127.19.52.66:46075)
0812 06:57:54.381533 (+ 376us) negotiation.cc:107] Waiting for socket to connect
0812 06:57:54.381622 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.52.126:42629: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":282,"thread_start_us":113,"threads_started":1}
I20260812 06:57:54.593276 4746 raft_consensus.cc:482] T ae36909a86394e4fada3c2cee01b3bb8 P 53890c7692834a24bd8919a9cbae000a [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 06:57:54.593423 4746 raft_consensus.cc:504] T ae36909a86394e4fada3c2cee01b3bb8 P 53890c7692834a24bd8919a9cbae000a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c25a0ae5f88b4400a21eb4c9085cd2c3" member_type: VOTER last_known_addr { host: "127.19.52.67" port: 46061 } } peers { permanent_uuid: "53890c7692834a24bd8919a9cbae000a" member_type: VOTER last_known_addr { host: "127.19.52.65" port: 43193 } } peers { permanent_uuid: "acc217e1270e4e56ac27f8bbe6ed20ba" member_type: VOTER last_known_addr { host: "127.19.52.66" port: 36889 } }
I20260812 06:57:54.593891 4746 leader_election.cc:288] T ae36909a86394e4fada3c2cee01b3bb8 P 53890c7692834a24bd8919a9cbae000a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers c25a0ae5f88b4400a21eb4c9085cd2c3 (127.19.52.67:46061), acc217e1270e4e56ac27f8bbe6ed20ba (127.19.52.66:36889)
W20260812 06:57:54.594779 4286 leader_election.cc:341] T ae36909a86394e4fada3c2cee01b3bb8 P 53890c7692834a24bd8919a9cbae000a [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer acc217e1270e4e56ac27f8bbe6ed20ba (127.19.52.66:36889): Not found: Tablet not found: ae36909a86394e4fada3c2cee01b3bb8
W20260812 06:57:54.594942 4289 leader_election.cc:341] T ae36909a86394e4fada3c2cee01b3bb8 P 53890c7692834a24bd8919a9cbae000a [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer c25a0ae5f88b4400a21eb4c9085cd2c3 (127.19.52.67:46061): Not found: Tablet not found: ae36909a86394e4fada3c2cee01b3bb8
I20260812 06:57:54.595028 4289 leader_election.cc:302] T ae36909a86394e4fada3c2cee01b3bb8 P 53890c7692834a24bd8919a9cbae000a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53890c7692834a24bd8919a9cbae000a; no voters: acc217e1270e4e56ac27f8bbe6ed20ba, c25a0ae5f88b4400a21eb4c9085cd2c3
I20260812 06:57:54.595242 4746 raft_consensus.cc:2726] T ae36909a86394e4fada3c2cee01b3bb8 P 53890c7692834a24bd8919a9cbae000a [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260812 06:57:54.608727 4747 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:57:54.608092 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.126:42629 (local address 127.19.52.67:49371)
0812 06:57:54.608415 (+ 323us) negotiation.cc:107] Waiting for socket to connect
0812 06:57:54.608493 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.52.126:42629: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":234,"thread_start_us":101,"threads_started":1}
I20260812 06:57:55.243254 4748 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:57:55.242762 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.126:42629 (local address 127.19.52.65:55523)
0812 06:57:55.243083 (+ 321us) negotiation.cc:107] Waiting for socket to connect
0812 06:57:55.243158 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.52.126:42629: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":236,"thread_start_us":104,"threads_started":1}
I20260812 06:57:55.383343 4749 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:57:55.382579 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.126:42629 (local address 127.19.52.66:43465)
0812 06:57:55.383163 (+ 584us) negotiation.cc:107] Waiting for socket to connect
0812 06:57:55.383234 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.52.126:42629: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":471,"thread_start_us":83,"threads_started":1}
I20260812 06:57:55.610374 4750 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:57:55.609793 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.126:42629 (local address 127.19.52.67:38919)
0812 06:57:55.610203 (+ 410us) negotiation.cc:107] Waiting for socket to connect
0812 06:57:55.610271 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.52.126:42629: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":266,"thread_start_us":134,"threads_started":1}
I20260812 06:57:56.245203 4751 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:57:56.244596 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.126:42629 (local address 127.19.52.65:55801)
0812 06:57:56.245026 (+ 430us) negotiation.cc:107] Waiting for socket to connect
0812 06:57:56.245098 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.52.126:42629: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":308,"thread_start_us":80,"threads_started":1}
I20260812 06:57:56.385094 4752 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:57:56.384476 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.126:42629 (local address 127.19.52.66:58343)
0812 06:57:56.384916 (+ 440us) negotiation.cc:107] Waiting for socket to connect
0812 06:57:56.384992 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.52.126:42629: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":351,"thread_start_us":198,"threads_started":1}
I20260812 06:57:56.612142 4754 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:57:56.611594 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.126:42629 (local address 127.19.52.67:37271)
0812 06:57:56.611924 (+ 330us) negotiation.cc:107] Waiting for socket to connect
0812 06:57:56.612041 (+ 117us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.52.126:42629: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":242,"thread_start_us":96,"threads_started":1}
I20260812 06:57:56.880980 4755 raft_consensus.cc:482] T ae36909a86394e4fada3c2cee01b3bb8 P 53890c7692834a24bd8919a9cbae000a [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 06:57:56.881134 4755 raft_consensus.cc:504] T ae36909a86394e4fada3c2cee01b3bb8 P 53890c7692834a24bd8919a9cbae000a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c25a0ae5f88b4400a21eb4c9085cd2c3" member_type: VOTER last_known_addr { host: "127.19.52.67" port: 46061 } } peers { permanent_uuid: "53890c7692834a24bd8919a9cbae000a" member_type: VOTER last_known_addr { host: "127.19.52.65" port: 43193 } } peers { permanent_uuid: "acc217e1270e4e56ac27f8bbe6ed20ba" member_type: VOTER last_known_addr { host: "127.19.52.66" port: 36889 } }
I20260812 06:57:56.881731 4755 leader_election.cc:288] T ae36909a86394e4fada3c2cee01b3bb8 P 53890c7692834a24bd8919a9cbae000a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers c25a0ae5f88b4400a21eb4c9085cd2c3 (127.19.52.67:46061), acc217e1270e4e56ac27f8bbe6ed20ba (127.19.52.66:36889)
W20260812 06:57:56.882687 4289 leader_election.cc:341] T ae36909a86394e4fada3c2cee01b3bb8 P 53890c7692834a24bd8919a9cbae000a [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer c25a0ae5f88b4400a21eb4c9085cd2c3 (127.19.52.67:46061): Not found: Tablet not found: ae36909a86394e4fada3c2cee01b3bb8
W20260812 06:57:56.882838 4286 leader_election.cc:341] T ae36909a86394e4fada3c2cee01b3bb8 P 53890c7692834a24bd8919a9cbae000a [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer acc217e1270e4e56ac27f8bbe6ed20ba (127.19.52.66:36889): Not found: Tablet not found: ae36909a86394e4fada3c2cee01b3bb8
I20260812 06:57:56.882970 4286 leader_election.cc:302] T ae36909a86394e4fada3c2cee01b3bb8 P 53890c7692834a24bd8919a9cbae000a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53890c7692834a24bd8919a9cbae000a; no voters: acc217e1270e4e56ac27f8bbe6ed20ba, c25a0ae5f88b4400a21eb4c9085cd2c3
I20260812 06:57:56.883136 4755 raft_consensus.cc:2726] T ae36909a86394e4fada3c2cee01b3bb8 P 53890c7692834a24bd8919a9cbae000a [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260812 06:57:57.246806 4756 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:57:57.246240 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.126:42629 (local address 127.19.52.65:49149)
0812 06:57:57.246605 (+ 365us) negotiation.cc:107] Waiting for socket to connect
0812 06:57:57.246679 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.52.126:42629: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":268,"thread_start_us":86,"threads_started":1}
I20260812 06:57:57.386631 4757 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:57:57.386110 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.126:42629 (local address 127.19.52.66:58691)
0812 06:57:57.386462 (+ 352us) negotiation.cc:107] Waiting for socket to connect
0812 06:57:57.386530 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.52.126:42629: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":239,"thread_start_us":86,"threads_started":1}
I20260812 06:57:57.613742 4758 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:57:57.613220 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.126:42629 (local address 127.19.52.67:38331)
0812 06:57:57.613571 (+ 351us) negotiation.cc:107] Waiting for socket to connect
0812 06:57:57.613642 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.52.126:42629: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":258,"thread_start_us":84,"threads_started":1}
I20260812 06:57:58.248425 4759 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:57:58.247904 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.126:42629 (local address 127.19.52.65:48439)
0812 06:57:58.248247 (+ 343us) negotiation.cc:107] Waiting for socket to connect
0812 06:57:58.248316 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.52.126:42629: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":259,"thread_start_us":87,"threads_started":1}
I20260812 06:57:58.388167 4760 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:57:58.387584 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.126:42629 (local address 127.19.52.66:57191)
0812 06:57:58.387994 (+ 410us) negotiation.cc:107] Waiting for socket to connect
0812 06:57:58.388066 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.52.126:42629: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":315,"thread_start_us":84,"threads_started":1}
I20260812 06:57:58.615288 4761 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:57:58.614719 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.126:42629 (local address 127.19.52.67:53049)
0812 06:57:58.615117 (+ 398us) negotiation.cc:107] Waiting for socket to connect
0812 06:57:58.615185 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.52.126:42629: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":307,"thread_start_us":85,"threads_started":1}
I20260812 06:57:59.249876 4762 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:57:59.249394 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.126:42629 (local address 127.19.52.65:41231)
0812 06:57:59.249708 (+ 314us) negotiation.cc:107] Waiting for socket to connect
0812 06:57:59.249773 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.52.126:42629: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":233,"thread_start_us":108,"threads_started":1}
I20260812 06:57:59.389690 4765 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:57:59.389217 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.126:42629 (local address 127.19.52.66:47765)
0812 06:57:59.389506 (+ 289us) negotiation.cc:107] Waiting for socket to connect
0812 06:57:59.389583 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.52.126:42629: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":203,"thread_start_us":84,"threads_started":1}
I20260812 06:57:59.616904 4766 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:57:59.616323 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.126:42629 (local address 127.19.52.67:37603)
0812 06:57:59.616725 (+ 402us) negotiation.cc:107] Waiting for socket to connect
0812 06:57:59.616796 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.52.126:42629: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":299,"thread_start_us":116,"threads_started":1}
I20260812 06:57:59.995275 4767 raft_consensus.cc:482] T ae36909a86394e4fada3c2cee01b3bb8 P 53890c7692834a24bd8919a9cbae000a [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 06:57:59.995427 4767 raft_consensus.cc:504] T ae36909a86394e4fada3c2cee01b3bb8 P 53890c7692834a24bd8919a9cbae000a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c25a0ae5f88b4400a21eb4c9085cd2c3" member_type: VOTER last_known_addr { host: "127.19.52.67" port: 46061 } } peers { permanent_uuid: "53890c7692834a24bd8919a9cbae000a" member_type: VOTER last_known_addr { host: "127.19.52.65" port: 43193 } } peers { permanent_uuid: "acc217e1270e4e56ac27f8bbe6ed20ba" member_type: VOTER last_known_addr { host: "127.19.52.66" port: 36889 } }
I20260812 06:57:59.995878 4767 leader_election.cc:288] T ae36909a86394e4fada3c2cee01b3bb8 P 53890c7692834a24bd8919a9cbae000a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers c25a0ae5f88b4400a21eb4c9085cd2c3 (127.19.52.67:46061), acc217e1270e4e56ac27f8bbe6ed20ba (127.19.52.66:36889)
W20260812 06:57:59.996867 4286 leader_election.cc:341] T ae36909a86394e4fada3c2cee01b3bb8 P 53890c7692834a24bd8919a9cbae000a [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer acc217e1270e4e56ac27f8bbe6ed20ba (127.19.52.66:36889): Not found: Tablet not found: ae36909a86394e4fada3c2cee01b3bb8
W20260812 06:57:59.997088 4289 leader_election.cc:341] T ae36909a86394e4fada3c2cee01b3bb8 P 53890c7692834a24bd8919a9cbae000a [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer c25a0ae5f88b4400a21eb4c9085cd2c3 (127.19.52.67:46061): Not found: Tablet not found: ae36909a86394e4fada3c2cee01b3bb8
I20260812 06:57:59.997184 4289 leader_election.cc:302] T ae36909a86394e4fada3c2cee01b3bb8 P 53890c7692834a24bd8919a9cbae000a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53890c7692834a24bd8919a9cbae000a; no voters: acc217e1270e4e56ac27f8bbe6ed20ba, c25a0ae5f88b4400a21eb4c9085cd2c3
I20260812 06:57:59.997361 4767 raft_consensus.cc:2726] T ae36909a86394e4fada3c2cee01b3bb8 P 53890c7692834a24bd8919a9cbae000a [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260812 06:58:00.251600 4768 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:58:00.251009 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.126:42629 (local address 127.19.52.65:59421)
0812 06:58:00.251336 (+ 327us) negotiation.cc:107] Waiting for socket to connect
0812 06:58:00.251402 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.52.126:42629: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":243,"thread_start_us":99,"threads_started":1}
I20260812 06:58:00.391036 4769 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:58:00.390532 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.126:42629 (local address 127.19.52.66:60673)
0812 06:58:00.390850 (+ 318us) negotiation.cc:107] Waiting for socket to connect
0812 06:58:00.390921 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.52.126:42629: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":233,"thread_start_us":102,"threads_started":1}
I20260812 06:58:00.618404 4771 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:58:00.617897 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.126:42629 (local address 127.19.52.67:33505)
0812 06:58:00.618237 (+ 340us) negotiation.cc:107] Waiting for socket to connect
0812 06:58:00.618301 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.52.126:42629: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":256,"thread_start_us":122,"threads_started":1}
I20260812 06:58:01.253221 4772 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:58:01.252739 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.126:42629 (local address 127.19.52.65:51707)
0812 06:58:01.253024 (+ 285us) negotiation.cc:107] Waiting for socket to connect
0812 06:58:01.253088 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.52.126:42629: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":206,"thread_start_us":82,"threads_started":1}
I20260812 06:58:01.392479 4773 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:58:01.392002 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.126:42629 (local address 127.19.52.66:53975)
0812 06:58:01.392308 (+ 306us) negotiation.cc:107] Waiting for socket to connect
0812 06:58:01.392376 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.52.126:42629: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":223,"thread_start_us":87,"threads_started":1}
I20260812 06:58:01.620126 4774 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:58:01.619423 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.126:42629 (local address 127.19.52.67:50621)
0812 06:58:01.619914 (+ 491us) negotiation.cc:107] Waiting for socket to connect
0812 06:58:01.619993 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.52.126:42629: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":395,"thread_start_us":88,"threads_started":1}
I20260812 06:58:02.254400 4775 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:58:02.253980 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.126:42629 (local address 127.19.52.65:37387)
0812 06:58:02.254242 (+ 262us) negotiation.cc:107] Waiting for socket to connect
0812 06:58:02.254303 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.52.126:42629: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":188,"thread_start_us":81,"threads_started":1}
I20260812 06:58:02.394112 4776 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:58:02.393559 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.126:42629 (local address 127.19.52.66:57169)
0812 06:58:02.393923 (+ 364us) negotiation.cc:107] Waiting for socket to connect
0812 06:58:02.393991 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.52.126:42629: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":276,"thread_start_us":107,"threads_started":1}
I20260812 06:58:02.621528 4777 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:58:02.621013 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.126:42629 (local address 127.19.52.67:37421)
0812 06:58:02.621325 (+ 312us) negotiation.cc:107] Waiting for socket to connect
0812 06:58:02.621401 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.52.126:42629: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":227,"thread_start_us":138,"threads_started":1}
I20260812 06:58:02.714962 4778 raft_consensus.cc:482] T ae36909a86394e4fada3c2cee01b3bb8 P 53890c7692834a24bd8919a9cbae000a [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 06:58:02.715152 4778 raft_consensus.cc:504] T ae36909a86394e4fada3c2cee01b3bb8 P 53890c7692834a24bd8919a9cbae000a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c25a0ae5f88b4400a21eb4c9085cd2c3" member_type: VOTER last_known_addr { host: "127.19.52.67" port: 46061 } } peers { permanent_uuid: "53890c7692834a24bd8919a9cbae000a" member_type: VOTER last_known_addr { host: "127.19.52.65" port: 43193 } } peers { permanent_uuid: "acc217e1270e4e56ac27f8bbe6ed20ba" member_type: VOTER last_known_addr { host: "127.19.52.66" port: 36889 } }
I20260812 06:58:02.715780 4778 leader_election.cc:288] T ae36909a86394e4fada3c2cee01b3bb8 P 53890c7692834a24bd8919a9cbae000a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers c25a0ae5f88b4400a21eb4c9085cd2c3 (127.19.52.67:46061), acc217e1270e4e56ac27f8bbe6ed20ba (127.19.52.66:36889)
W20260812 06:58:02.716688 4289 leader_election.cc:341] T ae36909a86394e4fada3c2cee01b3bb8 P 53890c7692834a24bd8919a9cbae000a [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer c25a0ae5f88b4400a21eb4c9085cd2c3 (127.19.52.67:46061): Not found: Tablet not found: ae36909a86394e4fada3c2cee01b3bb8
W20260812 06:58:02.716800 4286 leader_election.cc:341] T ae36909a86394e4fada3c2cee01b3bb8 P 53890c7692834a24bd8919a9cbae000a [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer acc217e1270e4e56ac27f8bbe6ed20ba (127.19.52.66:36889): Not found: Tablet not found: ae36909a86394e4fada3c2cee01b3bb8
I20260812 06:58:02.716877 4286 leader_election.cc:302] T ae36909a86394e4fada3c2cee01b3bb8 P 53890c7692834a24bd8919a9cbae000a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53890c7692834a24bd8919a9cbae000a; no voters: acc217e1270e4e56ac27f8bbe6ed20ba, c25a0ae5f88b4400a21eb4c9085cd2c3
I20260812 06:58:02.717145 4778 raft_consensus.cc:2726] T ae36909a86394e4fada3c2cee01b3bb8 P 53890c7692834a24bd8919a9cbae000a [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260812 06:58:03.256031 4779 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:58:03.255491 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.126:42629 (local address 127.19.52.65:45847)
0812 06:58:03.255844 (+ 353us) negotiation.cc:107] Waiting for socket to connect
0812 06:58:03.255916 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.52.126:42629: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":260,"thread_start_us":107,"threads_started":1}
I20260812 06:58:03.395514 4780 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:58:03.395010 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.126:42629 (local address 127.19.52.66:49311)
0812 06:58:03.395326 (+ 316us) negotiation.cc:107] Waiting for socket to connect
0812 06:58:03.395398 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.52.126:42629: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":230,"thread_start_us":91,"threads_started":1}
I20260812 06:58:03.622982 4782 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:58:03.622488 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.126:42629 (local address 127.19.52.67:59837)
0812 06:58:03.622790 (+ 302us) negotiation.cc:107] Waiting for socket to connect
0812 06:58:03.622856 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.52.126:42629: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":210,"thread_start_us":84,"threads_started":1}
I20260812 06:58:04.257543 4783 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:58:04.257054 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.126:42629 (local address 127.19.52.65:60299)
0812 06:58:04.257348 (+ 294us) negotiation.cc:107] Waiting for socket to connect
0812 06:58:04.257424 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.52.126:42629: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":212,"thread_start_us":89,"threads_started":1}
I20260812 06:58:04.397059 4784 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:58:04.396487 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.126:42629 (local address 127.19.52.66:50153)
0812 06:58:04.396862 (+ 375us) negotiation.cc:107] Waiting for socket to connect
0812 06:58:04.396947 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.52.126:42629: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":282,"thread_start_us":150,"threads_started":1}
I20260812 06:58:04.624341 4785 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:58:04.623813 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.126:42629 (local address 127.19.52.67:52965)
0812 06:58:04.624165 (+ 352us) negotiation.cc:107] Waiting for socket to connect
0812 06:58:04.624231 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.52.126:42629: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":226,"thread_start_us":96,"threads_started":1}
I20260812 06:58:05.259428 4786 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:58:05.258560 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.126:42629 (local address 127.19.52.65:58257)
0812 06:58:05.259072 (+ 512us) negotiation.cc:107] Waiting for socket to connect
0812 06:58:05.259178 (+ 106us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.52.126:42629: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":373,"thread_start_us":132,"threads_started":1}
I20260812 06:58:05.398326 4787 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:58:05.397884 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.126:42629 (local address 127.19.52.66:48561)
0812 06:58:05.398166 (+ 282us) negotiation.cc:107] Waiting for socket to connect
0812 06:58:05.398230 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.52.126:42629: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":202,"thread_start_us":79,"threads_started":1}
I20260812 06:58:05.626005 4789 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:58:05.625496 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.126:42629 (local address 127.19.52.67:46209)
0812 06:58:05.625836 (+ 340us) negotiation.cc:107] Waiting for socket to connect
0812 06:58:05.625902 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.52.126:42629: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":240,"thread_start_us":112,"threads_started":1}
I20260812 06:58:06.261224 4790 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:58:06.260702 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.126:42629 (local address 127.19.52.65:59707)
0812 06:58:06.261029 (+ 327us) negotiation.cc:107] Waiting for socket to connect
0812 06:58:06.261114 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.52.126:42629: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":230,"thread_start_us":100,"threads_started":1}
I20260812 06:58:06.399823 4791 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:58:06.399361 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.126:42629 (local address 127.19.52.66:41909)
0812 06:58:06.399656 (+ 295us) negotiation.cc:107] Waiting for socket to connect
0812 06:58:06.399724 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.52.126:42629: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":200,"thread_start_us":82,"threads_started":1}
I20260812 06:58:06.602010 19665 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu with pid 4232
I20260812 06:58:06.613801 19665 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu with pid 4424
I20260812 06:58:06.625586 19665 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu with pid 4573
2026-08-12T06:58:06Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedIPKICert (19833 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedTSK
Loading random data
Initializing database '/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517687384214-19665-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 06:58:06 dist-test-slave-65mx krb5kdc[4794](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 06:58:06 dist-test-slave-65mx krb5kdc[4794](info): set up 2 sockets
Aug 12 06:58:06 dist-test-slave-65mx krb5kdc[4794](info): commencing operation
krb5kdc: starting...
W20260812 06:58:07.787509 19665 mini_kdc.cc:121] Time spent starting KDC: real 1.135s user 0.000s sys 0.009s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517687384214-19665-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517687384214-19665-0/krb5kdc/test-user.keytab.
Aug 12 06:58:07 dist-test-slave-65mx krb5kdc[4794](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517887, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T06:58:07Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T06:58:07Z Disabled control of system clock
WARNING: no policy specified for kudu/127.19.52.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.52.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.52.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517687384214-19665-0/krb5kdc/kudu_127.19.52.126.keytab.
Entry for principal kudu/127.19.52.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517687384214-19665-0/krb5kdc/kudu_127.19.52.126.keytab.
I20260812 06:58:07.937142 19665 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu
/tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517687384214-19665-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517687384214-19665-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517687384214-19665-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517687384214-19665-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.52.126:36401
--webserver_interface=127.19.52.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517687384214-19665-0/krb5kdc/kudu_127.19.52.126.keytab
--principal=kudu/127.19.52.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.52.84:35215
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.19.52.126:36401
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517687384214-19665-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517687384214-19665-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517687384214-19665-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 06:58:08.060343 4811 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:58:08.060740 4811 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:58:08.060827 4811 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 06:58:08.065074 4811 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260812 06:58:08.065178 4811 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260812 06:58:08.065208 4811 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260812 06:58:08.065233 4811 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 06:58:08.065279 4811 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 06:58:08.065339 4811 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 06:58:08.065380 4811 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 06:58:08.065416 4811 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 06:58:08.071201 4811 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.52.84:35215
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517687384214-19665-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517687384214-19665-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.19.52.126:36401
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517687384214-19665-0/krb5kdc/kudu_127.19.52.126.keytab
--principal=kudu/127.19.52.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.52.126:36401
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517687384214-19665-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.19.52.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4811
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517687384214-19665-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 06:15:54 UTC on ccycloud.quasar-rnfzce.root.comops.site
build id 4581989
I20260812 06:58:08.072469 4811 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:58:08.073633 4811 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 06:58:08.083297 4820 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:58:08.083355 4811 server_base.cc:1034] running on GCE node
W20260812 06:58:08.083320 4817 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:58:08.083297 4818 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:58:08.084254 4811 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:58:08.085675 4811 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:58:08.086869 4811 hybrid_clock.cc:648] HybridClock initialized: now 1786517888086878 us; error 55 us; skew 500 ppm
Aug 12 06:58:08 dist-test-slave-65mx krb5kdc[4794](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517888, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.52.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 06:58:08.090113 4811 init.cc:377] Logged in from keytab as kudu/127.19.52.126@KRBTEST.COM (short username kudu)
I20260812 06:58:08.091312 4811 webserver.cc:460] Webserver started at http://127.19.52.126:40333/ using document root <none> and password file <none>
I20260812 06:58:08.091882 4811 fs_manager.cc:359] Metadata directory not provided
I20260812 06:58:08.091956 4811 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:58:08.092145 4811 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 06:58:08.094456 4811 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517687384214-19665-0/minicluster-data/master-0/data/instance:
uuid: "f4608298ae804b898d7b5b8077543057"
format_stamp: "Formatted at 2026-08-12 06:58:08 on dist-test-slave-65mx"
server_key: "e1431a0315a64cc0d3a002aa1a879aa3"
server_key_iv: "05eea69e240961fe06492fbfcc00dbef"
server_key_version: "encryptionkey@0"
I20260812 06:58:08.095067 4811 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517687384214-19665-0/minicluster-data/master-0/wal/instance:
uuid: "f4608298ae804b898d7b5b8077543057"
format_stamp: "Formatted at 2026-08-12 06:58:08 on dist-test-slave-65mx"
server_key: "e1431a0315a64cc0d3a002aa1a879aa3"
server_key_iv: "05eea69e240961fe06492fbfcc00dbef"
server_key_version: "encryptionkey@0"
I20260812 06:58:08.099555 4811 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.001s
I20260812 06:58:08.102383 4827 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 06:58:08.104039 4811 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260812 06:58:08.104208 4811 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517687384214-19665-0/minicluster-data/master-0/data,/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517687384214-19665-0/minicluster-data/master-0/wal
uuid: "f4608298ae804b898d7b5b8077543057"
format_stamp: "Formatted at 2026-08-12 06:58:08 on dist-test-slave-65mx"
server_key: "e1431a0315a64cc0d3a002aa1a879aa3"
server_key_iv: "05eea69e240961fe06492fbfcc00dbef"
server_key_version: "encryptionkey@0"
I20260812 06:58:08.104347 4811 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517687384214-19665-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517687384214-19665-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517687384214-19665-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:58:08.116714 4811 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 06:58:08.120177 4811 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 06:58:08.120400 4811 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:58:08.129029 4811 rpc_server.cc:307] RPC server started. Bound to: 127.19.52.126:36401
I20260812 06:58:08.129402 4888 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.52.126:36401 every 8 connection(s)
I20260812 06:58:08.130182 4811 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517687384214-19665-0/minicluster-data/master-0/data/info.pb
I20260812 06:58:08.133318 4889 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 06:58:08.136349 19665 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu as pid 4811
I20260812 06:58:08.136507 19665 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517687384214-19665-0/minicluster-data/master-0/wal/instance
I20260812 06:58:08.136847 19665 external_mini_cluster.cc:1428] Setting key cb6930293f8c66eaf98a288030adb089
I20260812 06:58:08.140028 4889 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f4608298ae804b898d7b5b8077543057: Bootstrap starting.
I20260812 06:58:08.143296 4889 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f4608298ae804b898d7b5b8077543057: Neither blocks nor log segments found. Creating new log.
I20260812 06:58:08.144392 4889 log.cc:824] T 00000000000000000000000000000000 P f4608298ae804b898d7b5b8077543057: Log is configured to *not* fsync() on all Append() calls
I20260812 06:58:08.147364 4889 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f4608298ae804b898d7b5b8077543057: No bootstrap required, opened a new log
Aug 12 06:58:08 dist-test-slave-65mx krb5kdc[4794](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517887, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.19.52.126@KRBTEST.COM
I20260812 06:58:08.150878 4889 raft_consensus.cc:348] T 00000000000000000000000000000000 P f4608298ae804b898d7b5b8077543057 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f4608298ae804b898d7b5b8077543057" member_type: VOTER last_known_addr { host: "127.19.52.126" port: 36401 } }
I20260812 06:58:08.151333 4889 raft_consensus.cc:374] T 00000000000000000000000000000000 P f4608298ae804b898d7b5b8077543057 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 06:58:08.151535 4889 raft_consensus.cc:729] T 00000000000000000000000000000000 P f4608298ae804b898d7b5b8077543057 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f4608298ae804b898d7b5b8077543057, State: Initialized, Role: FOLLOWER
I20260812 06:58:08.152324 4889 consensus_queue.cc:260] T 00000000000000000000000000000000 P f4608298ae804b898d7b5b8077543057 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f4608298ae804b898d7b5b8077543057" member_type: VOTER last_known_addr { host: "127.19.52.126" port: 36401 } }
I20260812 06:58:08.152518 4889 raft_consensus.cc:388] T 00000000000000000000000000000000 P f4608298ae804b898d7b5b8077543057 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 06:58:08.152673 4889 raft_consensus.cc:482] T 00000000000000000000000000000000 P f4608298ae804b898d7b5b8077543057 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 06:58:08.152781 4889 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f4608298ae804b898d7b5b8077543057 [term 0 FOLLOWER]: Advancing to term 1
I20260812 06:58:08.153820 4889 raft_consensus.cc:504] T 00000000000000000000000000000000 P f4608298ae804b898d7b5b8077543057 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f4608298ae804b898d7b5b8077543057" member_type: VOTER last_known_addr { host: "127.19.52.126" port: 36401 } }
I20260812 06:58:08.154246 4889 leader_election.cc:302] T 00000000000000000000000000000000 P f4608298ae804b898d7b5b8077543057 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f4608298ae804b898d7b5b8077543057; no voters:
I20260812 06:58:08.154631 4889 leader_election.cc:288] T 00000000000000000000000000000000 P f4608298ae804b898d7b5b8077543057 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 06:58:08.154764 4895 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f4608298ae804b898d7b5b8077543057 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 06:58:08.154994 4895 raft_consensus.cc:686] T 00000000000000000000000000000000 P f4608298ae804b898d7b5b8077543057 [term 1 LEADER]: Becoming Leader. State: Replica: f4608298ae804b898d7b5b8077543057, State: Running, Role: LEADER
I20260812 06:58:08.155395 4893 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:58:08.138482 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56250 (local address 127.19.52.126:36401)
0812 06:58:08.138953 (+ 471us) server_negotiation.cc:206] Beginning negotiation
0812 06:58:08.138971 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0812 06:58:08.139015 (+ 44us) server_negotiation.cc:407] Connection header received
0812 06:58:08.139682 (+ 667us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:58:08.139699 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:58:08.140057 (+ 358us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:58:08.140408 (+ 351us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:58:08.141397 (+ 989us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:58:08.142565 (+ 1168us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:58:08.143988 (+ 1423us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:58:08.144788 (+ 800us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:58:08.144970 (+ 182us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:58:08.148703 (+ 3733us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:58:08.148712 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:58:08.148809 (+ 97us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:58:08.148859 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:58:08.151885 (+ 3026us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:58:08.152477 (+ 592us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:58:08.152483 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:58:08.152488 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:58:08.152723 (+ 235us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:58:08.153025 (+ 302us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:58:08.153028 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:58:08.153030 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:58:08.153471 (+ 441us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:58:08.153680 (+ 209us) server_negotiation.cc:1035] Waiting for connection context
0812 06:58:08.153979 (+ 299us) server_negotiation.cc:299] Negotiation successful
0812 06:58:08.154308 (+ 329us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":315,"thread_start_us":153,"threads_started":1}
I20260812 06:58:08.155386 4895 consensus_queue.cc:237] T 00000000000000000000000000000000 P f4608298ae804b898d7b5b8077543057 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f4608298ae804b898d7b5b8077543057" member_type: VOTER last_known_addr { host: "127.19.52.126" port: 36401 } }
I20260812 06:58:08.156024 4889 sys_catalog.cc:564] T 00000000000000000000000000000000 P f4608298ae804b898d7b5b8077543057 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 06:58:08.157531 4897 sys_catalog.cc:455] T 00000000000000000000000000000000 P f4608298ae804b898d7b5b8077543057 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f4608298ae804b898d7b5b8077543057. Latest consensus state: current_term: 1 leader_uuid: "f4608298ae804b898d7b5b8077543057" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f4608298ae804b898d7b5b8077543057" member_type: VOTER last_known_addr { host: "127.19.52.126" port: 36401 } } }
I20260812 06:58:08.157675 4897 sys_catalog.cc:458] T 00000000000000000000000000000000 P f4608298ae804b898d7b5b8077543057 [sys.catalog]: This master's current role is: LEADER
I20260812 06:58:08.158025 4896 sys_catalog.cc:455] T 00000000000000000000000000000000 P f4608298ae804b898d7b5b8077543057 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f4608298ae804b898d7b5b8077543057" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f4608298ae804b898d7b5b8077543057" member_type: VOTER last_known_addr { host: "127.19.52.126" port: 36401 } } }
I20260812 06:58:08.158111 4896 sys_catalog.cc:458] T 00000000000000000000000000000000 P f4608298ae804b898d7b5b8077543057 [sys.catalog]: This master's current role is: LEADER
W20260812 06:58:08.161260 4914 catalog_manager.cc:1558] T 00000000000000000000000000000000 P f4608298ae804b898d7b5b8077543057: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 06:58:08.161374 4914 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 06:58:08.161444 4908 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 06:58:08.162189 4908 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 06:58:08.168718 4908 catalog_manager.cc:1347] Generated new cluster ID: 6dd4eb238b9f4adabe01ec004ec213af
I20260812 06:58:08.168819 4908 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 06:58:08.189617 4908 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 06:58:08.191134 4908 catalog_manager.cc:1504] Loading token signing keys...
I20260812 06:58:08.196836 4908 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f4608298ae804b898d7b5b8077543057: Generated new TSK 0
I20260812 06:58:08.197620 4908 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.19.52.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.52.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.52.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517687384214-19665-0/krb5kdc/kudu_127.19.52.65.keytab.
Entry for principal kudu/127.19.52.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517687384214-19665-0/krb5kdc/kudu_127.19.52.65.keytab.
I20260812 06:58:08.236701 19665 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu
/tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517687384214-19665-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517687384214-19665-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517687384214-19665-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517687384214-19665-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.52.65:0
--local_ip_for_outbound_sockets=127.19.52.65
--webserver_interface=127.19.52.65
--webserver_port=0
--tserver_master_addrs=127.19.52.126:36401
--keytab_file=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517687384214-19665-0/krb5kdc/kudu_127.19.52.65.keytab
--principal=kudu/127.19.52.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.52.84:35215
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517687384214-19665-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517687384214-19665-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517687384214-19665-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 06:58:08.386904 4920 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:58:08.387261 4920 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:58:08.387310 4920 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 06:58:08.392393 4920 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 06:58:08.392656 4920 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 06:58:08.392776 4920 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.52.65
I20260812 06:58:08.398881 4920 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.52.84:35215
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517687384214-19665-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517687384214-19665-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517687384214-19665-0/krb5kdc/kudu_127.19.52.65.keytab
--principal=kudu/127.19.52.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.52.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517687384214-19665-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.19.52.65
--webserver_port=0
--tserver_master_addrs=127.19.52.126:36401
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4920
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.52.65
--log_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517687384214-19665-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 06:15:54 UTC on ccycloud.quasar-rnfzce.root.comops.site
build id 4581989
I20260812 06:58:08.400067 4920 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:58:08.401235 4920 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 06:58:08.412057 4930 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:58:08.412174 4927 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:58:08.412057 4928 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:58:08.412864 4920 server_base.cc:1034] running on GCE node
I20260812 06:58:08.413379 4920 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:58:08.414117 4920 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:58:08.415315 4920 hybrid_clock.cc:648] HybridClock initialized: now 1786517888415289 us; error 43 us; skew 500 ppm
Aug 12 06:58:08 dist-test-slave-65mx krb5kdc[4794](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517888, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.52.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 06:58:08.418378 4920 init.cc:377] Logged in from keytab as kudu/127.19.52.65@KRBTEST.COM (short username kudu)
I20260812 06:58:08.419341 4920 webserver.cc:460] Webserver started at http://127.19.52.65:44717/ using document root <none> and password file <none>
I20260812 06:58:08.419925 4920 fs_manager.cc:359] Metadata directory not provided
I20260812 06:58:08.419993 4920 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:58:08.420172 4920 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 06:58:08.422842 4920 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517687384214-19665-0/minicluster-data/ts-0/data/instance:
uuid: "aa4029c643b448ad9e197e92749c8fe6"
format_stamp: "Formatted at 2026-08-12 06:58:08 on dist-test-slave-65mx"
server_key: "8850f5c2ac7fdd0cd582238231808ff3"
server_key_iv: "2ac23c925edf745076c39d522dc5cf21"
server_key_version: "encryptionkey@0"
I20260812 06:58:08.423422 4920 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517687384214-19665-0/minicluster-data/ts-0/wal/instance:
uuid: "aa4029c643b448ad9e197e92749c8fe6"
format_stamp: "Formatted at 2026-08-12 06:58:08 on dist-test-slave-65mx"
server_key: "8850f5c2ac7fdd0cd582238231808ff3"
server_key_iv: "2ac23c925edf745076c39d522dc5cf21"
server_key_version: "encryptionkey@0"
I20260812 06:58:08.428139 4920 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.002s
I20260812 06:58:08.431267 4939 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 06:58:08.432932 4920 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260812 06:58:08.433126 4920 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517687384214-19665-0/minicluster-data/ts-0/data,/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517687384214-19665-0/minicluster-data/ts-0/wal
uuid: "aa4029c643b448ad9e197e92749c8fe6"
format_stamp: "Formatted at 2026-08-12 06:58:08 on dist-test-slave-65mx"
server_key: "8850f5c2ac7fdd0cd582238231808ff3"
server_key_iv: "2ac23c925edf745076c39d522dc5cf21"
server_key_version: "encryptionkey@0"
I20260812 06:58:08.433269 4920 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517687384214-19665-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517687384214-19665-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517687384214-19665-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:58:08.460184 4920 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 06:58:08.463642 4920 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 06:58:08.463871 4920 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:58:08.464588 4920 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 06:58:08.465528 4920 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 06:58:08.465580 4920 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 06:58:08.465624 4920 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 06:58:08.465641 4920 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 06:58:08.476051 4920 rpc_server.cc:307] RPC server started. Bound to: 127.19.52.65:41283
I20260812 06:58:08.476161 5072 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.52.65:41283 every 8 connection(s)
I20260812 06:58:08.477209 4920 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517687384214-19665-0/minicluster-data/ts-0/data/info.pb
I20260812 06:58:08.487213 19665 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu as pid 4920
I20260812 06:58:08.487375 19665 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517687384214-19665-0/minicluster-data/ts-0/wal/instance
I20260812 06:58:08.488176 19665 external_mini_cluster.cc:1428] Setting key a27adfe88655f726ffa809a81baaa5d9
Aug 12 06:58:08 dist-test-slave-65mx krb5kdc[4794](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517888, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.52.65@KRBTEST.COM for kudu/127.19.52.126@KRBTEST.COM
I20260812 06:58:08.494325 4893 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:58:08.479248 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.52.65:55769 (local address 127.19.52.126:36401)
0812 06:58:08.479388 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0812 06:58:08.479392 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 06:58:08.480618 (+ 1226us) server_negotiation.cc:407] Connection header received
0812 06:58:08.482376 (+ 1758us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:58:08.482379 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:58:08.482461 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:58:08.482571 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:58:08.484237 (+ 1666us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:58:08.484877 (+ 640us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:58:08.486185 (+ 1308us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:58:08.486525 (+ 340us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:58:08.486691 (+ 166us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:58:08.489580 (+ 2889us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:58:08.489584 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:58:08.489587 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:58:08.489615 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:58:08.491279 (+ 1664us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:58:08.492114 (+ 835us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:58:08.492120 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:58:08.492124 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:58:08.492295 (+ 171us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:58:08.492814 (+ 519us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:58:08.492820 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:58:08.492825 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:58:08.493164 (+ 339us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:58:08.493294 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0812 06:58:08.493864 (+ 570us) server_negotiation.cc:299] Negotiation successful
0812 06:58:08.494054 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260812 06:58:08.495015 5076 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:58:08.479668 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.126:36401 (local address 127.19.52.65:55769)
0812 06:58:08.480284 (+ 616us) negotiation.cc:107] Waiting for socket to connect
0812 06:58:08.480335 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0812 06:58:08.482096 (+ 1761us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:58:08.482791 (+ 695us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:58:08.482799 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:58:08.483438 (+ 639us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:58:08.484026 (+ 588us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:58:08.484044 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:58:08.485061 (+ 1017us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:58:08.485064 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:58:08.486035 (+ 971us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:58:08.486047 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:58:08.486752 (+ 705us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:58:08.486756 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:58:08.486961 (+ 205us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:58:08.487675 (+ 714us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 06:58:08.487693 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:58:08.489376 (+ 1683us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 06:58:08.491424 (+ 2048us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:58:08.491435 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:58:08.491458 (+ 23us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:58:08.491937 (+ 479us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:58:08.492437 (+ 500us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:58:08.492440 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:58:08.492451 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:58:08.492694 (+ 243us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:58:08.493291 (+ 597us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 06:58:08.493297 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 06:58:08.493579 (+ 282us) client_negotiation.cc:769] Sending connection context
0812 06:58:08.493753 (+ 174us) client_negotiation.cc:240] Negotiation successful
0812 06:58:08.494057 (+ 304us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":368,"thread_start_us":135,"threads_started":1}
I20260812 06:58:08.496618 5073 heartbeater.cc:347] Connected to a master server at 127.19.52.126:36401
I20260812 06:58:08.496913 5073 heartbeater.cc:464] Registering TS with master...
I20260812 06:58:08.497471 5073 heartbeater.cc:511] Master 127.19.52.126:36401 requested a full tablet report, sending...
I20260812 06:58:08.499246 4846 ts_manager.cc:194] Registered new tserver with Master: aa4029c643b448ad9e197e92749c8fe6 (127.19.52.65:41283)
WARNING: no policy specified for kudu/127.19.52.66@KRBTEST.COM; defaulting to no policy
I20260812 06:58:08.500682 4846 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.52.65@KRBTEST.COM'} at 127.19.52.65:55769
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.52.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.52.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517687384214-19665-0/krb5kdc/kudu_127.19.52.66.keytab.
Entry for principal kudu/127.19.52.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517687384214-19665-0/krb5kdc/kudu_127.19.52.66.keytab.
I20260812 06:58:08.525784 19665 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu
/tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517687384214-19665-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517687384214-19665-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517687384214-19665-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517687384214-19665-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.52.66:0
--local_ip_for_outbound_sockets=127.19.52.66
--webserver_interface=127.19.52.66
--webserver_port=0
--tserver_master_addrs=127.19.52.126:36401
--keytab_file=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517687384214-19665-0/krb5kdc/kudu_127.19.52.66.keytab
--principal=kudu/127.19.52.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.52.84:35215
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517687384214-19665-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517687384214-19665-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517687384214-19665-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 06:58:08.656973 5079 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:58:08.657390 5079 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:58:08.657478 5079 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 06:58:08.662012 5079 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 06:58:08.662119 5079 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 06:58:08.662218 5079 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.52.66
I20260812 06:58:08.668623 5079 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.52.84:35215
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517687384214-19665-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517687384214-19665-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517687384214-19665-0/krb5kdc/kudu_127.19.52.66.keytab
--principal=kudu/127.19.52.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.52.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517687384214-19665-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.19.52.66
--webserver_port=0
--tserver_master_addrs=127.19.52.126:36401
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5079
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.52.66
--log_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517687384214-19665-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 06:15:54 UTC on ccycloud.quasar-rnfzce.root.comops.site
build id 4581989
I20260812 06:58:08.669883 5079 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:58:08.671029 5079 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 06:58:08.682096 5089 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:58:08.682091 5085 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:58:08.682320 5079 server_base.cc:1034] running on GCE node
W20260812 06:58:08.682142 5086 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:58:08.683014 5079 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:58:08.683769 5079 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:58:08.684968 5079 hybrid_clock.cc:648] HybridClock initialized: now 1786517888684942 us; error 43 us; skew 500 ppm
Aug 12 06:58:08 dist-test-slave-65mx krb5kdc[4794](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517888, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.52.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 06:58:08.688203 5079 init.cc:377] Logged in from keytab as kudu/127.19.52.66@KRBTEST.COM (short username kudu)
I20260812 06:58:08.689523 5079 webserver.cc:460] Webserver started at http://127.19.52.66:37613/ using document root <none> and password file <none>
I20260812 06:58:08.690183 5079 fs_manager.cc:359] Metadata directory not provided
I20260812 06:58:08.690261 5079 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:58:08.690498 5079 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 06:58:08.693239 5079 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517687384214-19665-0/minicluster-data/ts-1/data/instance:
uuid: "25ef252218c24d6cae8e3f5efff644ea"
format_stamp: "Formatted at 2026-08-12 06:58:08 on dist-test-slave-65mx"
server_key: "8528faeac61ece0467e7f9d0be88de1a"
server_key_iv: "957fc3fb0af5270c72c4d5ba7c779a04"
server_key_version: "encryptionkey@0"
I20260812 06:58:08.694166 5079 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517687384214-19665-0/minicluster-data/ts-1/wal/instance:
uuid: "25ef252218c24d6cae8e3f5efff644ea"
format_stamp: "Formatted at 2026-08-12 06:58:08 on dist-test-slave-65mx"
server_key: "8528faeac61ece0467e7f9d0be88de1a"
server_key_iv: "957fc3fb0af5270c72c4d5ba7c779a04"
server_key_version: "encryptionkey@0"
I20260812 06:58:08.699272 5079 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.004s
I20260812 06:58:08.702222 5097 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 06:58:08.703727 5079 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260812 06:58:08.703987 5079 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517687384214-19665-0/minicluster-data/ts-1/data,/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517687384214-19665-0/minicluster-data/ts-1/wal
uuid: "25ef252218c24d6cae8e3f5efff644ea"
format_stamp: "Formatted at 2026-08-12 06:58:08 on dist-test-slave-65mx"
server_key: "8528faeac61ece0467e7f9d0be88de1a"
server_key_iv: "957fc3fb0af5270c72c4d5ba7c779a04"
server_key_version: "encryptionkey@0"
I20260812 06:58:08.704142 5079 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517687384214-19665-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517687384214-19665-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517687384214-19665-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:58:08.715754 5079 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 06:58:08.719341 5079 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 06:58:08.719774 5079 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:58:08.720486 5079 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 06:58:08.721501 5079 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 06:58:08.721695 5079 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 06:58:08.721813 5079 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 06:58:08.721868 5079 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 06:58:08.731719 5079 rpc_server.cc:307] RPC server started. Bound to: 127.19.52.66:34423
I20260812 06:58:08.731737 5226 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.52.66:34423 every 8 connection(s)
I20260812 06:58:08.732766 5079 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517687384214-19665-0/minicluster-data/ts-1/data/info.pb
I20260812 06:58:08.735728 19665 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu as pid 5079
I20260812 06:58:08.735890 19665 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517687384214-19665-0/minicluster-data/ts-1/wal/instance
I20260812 06:58:08.736250 19665 external_mini_cluster.cc:1428] Setting key af02d0c0ec34e42e4dcdd3fa94a2f430
Aug 12 06:58:08 dist-test-slave-65mx krb5kdc[4794](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517888, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.52.66@KRBTEST.COM for kudu/127.19.52.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.19.52.67@KRBTEST.COM; defaulting to no policy
I20260812 06:58:08.752667 4893 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:58:08.734764 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.52.66:35013 (local address 127.19.52.126:36401)
0812 06:58:08.734944 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0812 06:58:08.734949 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 06:58:08.735956 (+ 1007us) server_negotiation.cc:407] Connection header received
0812 06:58:08.738223 (+ 2267us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:58:08.738227 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:58:08.738284 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:58:08.738390 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:58:08.740200 (+ 1810us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:58:08.741070 (+ 870us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:58:08.742854 (+ 1784us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:58:08.743292 (+ 438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:58:08.743427 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:58:08.746857 (+ 3430us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:58:08.746863 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:58:08.746866 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:58:08.746905 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:58:08.748943 (+ 2038us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:58:08.749803 (+ 860us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:58:08.749807 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:58:08.749810 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:58:08.749927 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:58:08.751058 (+ 1131us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:58:08.751061 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:58:08.751062 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:58:08.751300 (+ 238us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:58:08.751436 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0812 06:58:08.752255 (+ 819us) server_negotiation.cc:299] Negotiation successful
0812 06:58:08.752397 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.52.67@KRBTEST.COM" created.
I20260812 06:58:08.753811 5229 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:58:08.735242 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.126:36401 (local address 127.19.52.66:35013)
0812 06:58:08.735763 (+ 521us) negotiation.cc:107] Waiting for socket to connect
0812 06:58:08.735822 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0812 06:58:08.737924 (+ 2102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:58:08.738621 (+ 697us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:58:08.738632 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:58:08.739219 (+ 587us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:58:08.739994 (+ 775us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:58:08.740020 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:58:08.741266 (+ 1246us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:58:08.741271 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:58:08.742720 (+ 1449us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:58:08.742732 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:58:08.743475 (+ 743us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:58:08.743480 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:58:08.743647 (+ 167us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:58:08.744399 (+ 752us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 06:58:08.744506 (+ 107us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:58:08.746630 (+ 2124us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 06:58:08.749136 (+ 2506us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:58:08.749142 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:58:08.749158 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:58:08.749668 (+ 510us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:58:08.750186 (+ 518us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:58:08.750192 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:58:08.750194 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:58:08.750775 (+ 581us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:58:08.751579 (+ 804us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 06:58:08.751586 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 06:58:08.751931 (+ 345us) client_negotiation.cc:769] Sending connection context
0812 06:58:08.752144 (+ 213us) client_negotiation.cc:240] Negotiation successful
0812 06:58:08.752706 (+ 562us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":306,"thread_start_us":150,"threads_started":1}
I20260812 06:58:08.755156 5227 heartbeater.cc:347] Connected to a master server at 127.19.52.126:36401
I20260812 06:58:08.755450 5227 heartbeater.cc:464] Registering TS with master...
I20260812 06:58:08.756096 5227 heartbeater.cc:511] Master 127.19.52.126:36401 requested a full tablet report, sending...
I20260812 06:58:08.757723 4846 ts_manager.cc:194] Registered new tserver with Master: 25ef252218c24d6cae8e3f5efff644ea (127.19.52.66:34423)
I20260812 06:58:08.758811 4846 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.52.66@KRBTEST.COM'} at 127.19.52.66:35013
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.52.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517687384214-19665-0/krb5kdc/kudu_127.19.52.67.keytab.
Entry for principal kudu/127.19.52.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517687384214-19665-0/krb5kdc/kudu_127.19.52.67.keytab.
I20260812 06:58:08.776142 19665 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu
/tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517687384214-19665-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517687384214-19665-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517687384214-19665-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517687384214-19665-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.52.67:0
--local_ip_for_outbound_sockets=127.19.52.67
--webserver_interface=127.19.52.67
--webserver_port=0
--tserver_master_addrs=127.19.52.126:36401
--keytab_file=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517687384214-19665-0/krb5kdc/kudu_127.19.52.67.keytab
--principal=kudu/127.19.52.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.52.84:35215
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517687384214-19665-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517687384214-19665-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517687384214-19665-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 06:58:08.901751 5233 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:58:08.902056 5233 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:58:08.902104 5233 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 06:58:08.906847 5233 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 06:58:08.906941 5233 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 06:58:08.907017 5233 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.52.67
I20260812 06:58:08.913271 5233 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.52.84:35215
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517687384214-19665-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517687384214-19665-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517687384214-19665-0/krb5kdc/kudu_127.19.52.67.keytab
--principal=kudu/127.19.52.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.52.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517687384214-19665-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.19.52.67
--webserver_port=0
--tserver_master_addrs=127.19.52.126:36401
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5233
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.52.67
--log_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517687384214-19665-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 06:15:54 UTC on ccycloud.quasar-rnfzce.root.comops.site
build id 4581989
I20260812 06:58:08.914433 5233 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:58:08.915477 5233 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 06:58:08.925971 5240 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:58:08.925938 5243 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:58:08.925938 5241 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:58:08.926234 5233 server_base.cc:1034] running on GCE node
I20260812 06:58:08.926748 5233 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:58:08.927511 5233 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:58:08.928740 5233 hybrid_clock.cc:648] HybridClock initialized: now 1786517888928698 us; error 54 us; skew 500 ppm
Aug 12 06:58:08 dist-test-slave-65mx krb5kdc[4794](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517888, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.52.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 06:58:08.932251 5233 init.cc:377] Logged in from keytab as kudu/127.19.52.67@KRBTEST.COM (short username kudu)
I20260812 06:58:08.933344 5233 webserver.cc:460] Webserver started at http://127.19.52.67:41963/ using document root <none> and password file <none>
I20260812 06:58:08.933975 5233 fs_manager.cc:359] Metadata directory not provided
I20260812 06:58:08.934073 5233 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:58:08.934336 5233 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 06:58:08.936621 5233 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517687384214-19665-0/minicluster-data/ts-2/data/instance:
uuid: "666220fa0cdf403dad047d3011fed16f"
format_stamp: "Formatted at 2026-08-12 06:58:08 on dist-test-slave-65mx"
server_key: "76e41badd775cd19f48c41252a471152"
server_key_iv: "8b7221af01d65e7fcbc265f760f1f591"
server_key_version: "encryptionkey@0"
I20260812 06:58:08.937719 5233 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517687384214-19665-0/minicluster-data/ts-2/wal/instance:
uuid: "666220fa0cdf403dad047d3011fed16f"
format_stamp: "Formatted at 2026-08-12 06:58:08 on dist-test-slave-65mx"
server_key: "76e41badd775cd19f48c41252a471152"
server_key_iv: "8b7221af01d65e7fcbc265f760f1f591"
server_key_version: "encryptionkey@0"
I20260812 06:58:08.942466 5233 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.003s
I20260812 06:58:08.945752 5252 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 06:58:08.947333 5233 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.004s
I20260812 06:58:08.947513 5233 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517687384214-19665-0/minicluster-data/ts-2/data,/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517687384214-19665-0/minicluster-data/ts-2/wal
uuid: "666220fa0cdf403dad047d3011fed16f"
format_stamp: "Formatted at 2026-08-12 06:58:08 on dist-test-slave-65mx"
server_key: "76e41badd775cd19f48c41252a471152"
server_key_iv: "8b7221af01d65e7fcbc265f760f1f591"
server_key_version: "encryptionkey@0"
I20260812 06:58:08.947654 5233 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517687384214-19665-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517687384214-19665-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517687384214-19665-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:58:08.969630 5233 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 06:58:08.974313 5233 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 06:58:08.974704 5233 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:58:08.975473 5233 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 06:58:08.976410 5233 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 06:58:08.976461 5233 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 06:58:08.976503 5233 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 06:58:08.976543 5233 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 06:58:08.986500 5233 rpc_server.cc:307] RPC server started. Bound to: 127.19.52.67:35465
I20260812 06:58:08.986586 5384 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.52.67:35465 every 8 connection(s)
I20260812 06:58:08.987850 5233 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517687384214-19665-0/minicluster-data/ts-2/data/info.pb
I20260812 06:58:08.995502 19665 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu as pid 5233
I20260812 06:58:08.995639 19665 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517687384214-19665-0/minicluster-data/ts-2/wal/instance
I20260812 06:58:08.995942 19665 external_mini_cluster.cc:1428] Setting key 5cce3187fd5fe733dea66b0f006d3b78
Aug 12 06:58:08 dist-test-slave-65mx krb5kdc[4794](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517888, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.52.67@KRBTEST.COM for kudu/127.19.52.126@KRBTEST.COM
I20260812 06:58:09.005379 4893 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:58:08.990796 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.52.67:39977 (local address 127.19.52.126:36401)
0812 06:58:08.990957 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0812 06:58:08.990961 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 06:58:08.991961 (+ 1000us) server_negotiation.cc:407] Connection header received
0812 06:58:08.993480 (+ 1519us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:58:08.993483 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:58:08.993552 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:58:08.993669 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:58:08.994999 (+ 1330us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:58:08.995840 (+ 841us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:58:08.997715 (+ 1875us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:58:08.997949 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:58:08.998034 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:58:09.001401 (+ 3367us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:58:09.001405 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:58:09.001408 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:58:09.001438 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:58:09.003012 (+ 1574us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:58:09.003653 (+ 641us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:58:09.003656 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:58:09.003658 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:58:09.003739 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:58:09.004118 (+ 379us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:58:09.004121 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:58:09.004123 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:58:09.004328 (+ 205us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:58:09.004437 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0812 06:58:09.005056 (+ 619us) server_negotiation.cc:299] Negotiation successful
0812 06:58:09.005203 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260812 06:58:09.006305 5388 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:58:08.991205 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.126:36401 (local address 127.19.52.67:39977)
0812 06:58:08.991759 (+ 554us) negotiation.cc:107] Waiting for socket to connect
0812 06:58:08.991808 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0812 06:58:08.993227 (+ 1419us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:58:08.993867 (+ 640us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:58:08.993875 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:58:08.994298 (+ 423us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:58:08.994823 (+ 525us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:58:08.994838 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:58:08.996051 (+ 1213us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:58:08.996056 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:58:08.997594 (+ 1538us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:58:08.997608 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:58:08.998116 (+ 508us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:58:08.998120 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:58:08.998341 (+ 221us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:58:08.999061 (+ 720us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 06:58:08.999087 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:58:09.001140 (+ 2053us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 06:58:09.003180 (+ 2040us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:58:09.003187 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:58:09.003212 (+ 25us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:58:09.003526 (+ 314us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:58:09.003837 (+ 311us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:58:09.003840 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:58:09.003842 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:58:09.004012 (+ 170us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:58:09.004438 (+ 426us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 06:58:09.004448 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 06:58:09.004794 (+ 346us) client_negotiation.cc:769] Sending connection context
0812 06:58:09.004995 (+ 201us) client_negotiation.cc:240] Negotiation successful
0812 06:58:09.005384 (+ 389us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":338,"thread_start_us":146,"threads_started":1}
I20260812 06:58:09.007629 5385 heartbeater.cc:347] Connected to a master server at 127.19.52.126:36401
I20260812 06:58:09.007916 5385 heartbeater.cc:464] Registering TS with master...
I20260812 06:58:09.008632 5385 heartbeater.cc:511] Master 127.19.52.126:36401 requested a full tablet report, sending...
I20260812 06:58:09.009827 4846 ts_manager.cc:194] Registered new tserver with Master: 666220fa0cdf403dad047d3011fed16f (127.19.52.67:35465)
I20260812 06:58:09.010644 19665 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 06:58:09.010651 4846 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.52.67@KRBTEST.COM'} at 127.19.52.67:39977
I20260812 06:58:09.020920 4893 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:58:09.013167 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56264 (local address 127.19.52.126:36401)
0812 06:58:09.013300 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0812 06:58:09.013303 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 06:58:09.013502 (+ 199us) server_negotiation.cc:407] Connection header received
0812 06:58:09.013622 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:58:09.013627 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:58:09.013666 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:58:09.013736 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:58:09.014448 (+ 712us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:58:09.014962 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:58:09.015784 (+ 822us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:58:09.015995 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:58:09.016082 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:58:09.017554 (+ 1472us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:58:09.017557 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:58:09.017559 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:58:09.017585 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:58:09.018952 (+ 1367us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:58:09.019388 (+ 436us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:58:09.019390 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:58:09.019391 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:58:09.019582 (+ 191us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:58:09.019873 (+ 291us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:58:09.019876 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:58:09.019877 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:58:09.020122 (+ 245us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:58:09.020233 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0812 06:58:09.020680 (+ 447us) server_negotiation.cc:299] Negotiation successful
0812 06:58:09.020793 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260812 06:58:09.024281 4846 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56264:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 06:58:09.027643 4846 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 06:58:09.044727 5400 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:58:09.038415 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.67:35465 (local address 127.0.0.1:44480)
0812 06:58:09.039439 (+ 1024us) negotiation.cc:107] Waiting for socket to connect
0812 06:58:09.039455 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0812 06:58:09.039562 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:58:09.040151 (+ 589us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:58:09.040154 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:58:09.040174 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 06:58:09.040282 (+ 108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:58:09.040290 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:58:09.041386 (+ 1096us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:58:09.041390 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:58:09.042652 (+ 1262us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:58:09.042665 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:58:09.043760 (+ 1095us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:58:09.043763 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:58:09.044069 (+ 306us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:58:09.044239 (+ 170us) client_negotiation.cc:769] Sending connection context
0812 06:58:09.044334 (+ 95us) client_negotiation.cc:240] Negotiation successful
0812 06:58:09.044505 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":811,"mutex_wait_us":100,"thread_start_us":114,"threads_started":1}
I20260812 06:58:09.044772 5399 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:58:09.037863 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42240 (local address 127.19.52.66:34423)
0812 06:58:09.038272 (+ 409us) server_negotiation.cc:206] Beginning negotiation
0812 06:58:09.038287 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0812 06:58:09.038734 (+ 447us) server_negotiation.cc:407] Connection header received
0812 06:58:09.039016 (+ 282us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:58:09.039024 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:58:09.039258 (+ 234us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:58:09.039394 (+ 136us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 06:58:09.039963 (+ 569us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:58:09.041190 (+ 1227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:58:09.042810 (+ 1620us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:58:09.043554 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:58:09.043633 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:58:09.043810 (+ 177us) server_negotiation.cc:1035] Waiting for connection context
0812 06:58:09.044396 (+ 586us) server_negotiation.cc:299] Negotiation successful
0812 06:58:09.044554 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":293,"thread_start_us":166,"threads_started":1}
I20260812 06:58:09.044739 5401 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:58:09.037931 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.65:41283 (local address 127.0.0.1:51582)
0812 06:58:09.038909 (+ 978us) negotiation.cc:107] Waiting for socket to connect
0812 06:58:09.038926 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0812 06:58:09.039007 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:58:09.040326 (+ 1319us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:58:09.040329 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:58:09.040349 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 06:58:09.040435 (+ 86us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:58:09.040440 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:58:09.041986 (+ 1546us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:58:09.041990 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:58:09.043094 (+ 1104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:58:09.043105 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:58:09.044267 (+ 1162us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:58:09.044274 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:58:09.044405 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:58:09.044427 (+ 22us) client_negotiation.cc:769] Sending connection context
0812 06:58:09.044522 (+ 95us) client_negotiation.cc:240] Negotiation successful
0812 06:58:09.044625 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":901,"thread_start_us":112,"threads_started":1}
I20260812 06:58:09.044842 5403 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:58:09.039264 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51582 (local address 127.19.52.65:41283)
0812 06:58:09.039870 (+ 606us) server_negotiation.cc:206] Beginning negotiation
0812 06:58:09.039879 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 06:58:09.039906 (+ 27us) server_negotiation.cc:407] Connection header received
0812 06:58:09.039982 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:58:09.039987 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:58:09.040166 (+ 179us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:58:09.040624 (+ 458us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 06:58:09.040774 (+ 150us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:58:09.041849 (+ 1075us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:58:09.043240 (+ 1391us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:58:09.044112 (+ 872us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:58:09.044232 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:58:09.044394 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0812 06:58:09.044593 (+ 199us) server_negotiation.cc:299] Negotiation successful
0812 06:58:09.044698 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":492,"thread_start_us":82,"threads_started":1}
I20260812 06:58:09.045444 5398 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:58:09.037674 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.66:34423 (local address 127.0.0.1:42240)
0812 06:58:09.038561 (+ 887us) negotiation.cc:107] Waiting for socket to connect
0812 06:58:09.038621 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0812 06:58:09.038799 (+ 178us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:58:09.039639 (+ 840us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:58:09.039645 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:58:09.039668 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 06:58:09.039818 (+ 150us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:58:09.039825 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:58:09.041338 (+ 1513us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:58:09.041342 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:58:09.042660 (+ 1318us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:58:09.042671 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:58:09.044017 (+ 1346us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:58:09.044021 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:58:09.044115 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:58:09.044180 (+ 65us) client_negotiation.cc:769] Sending connection context
0812 06:58:09.045214 (+ 1034us) client_negotiation.cc:240] Negotiation successful
0812 06:58:09.045307 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":676,"mutex_wait_us":26,"thread_start_us":94,"threads_started":1}
I20260812 06:58:09.045558 5402 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:58:09.038804 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44480 (local address 127.19.52.67:35465)
0812 06:58:09.039258 (+ 454us) server_negotiation.cc:206] Beginning negotiation
0812 06:58:09.039272 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0812 06:58:09.039657 (+ 385us) server_negotiation.cc:407] Connection header received
0812 06:58:09.039758 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:58:09.039764 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:58:09.039953 (+ 189us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:58:09.040180 (+ 227us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 06:58:09.040386 (+ 206us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:58:09.041225 (+ 839us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:58:09.042805 (+ 1580us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:58:09.043618 (+ 813us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:58:09.043735 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:58:09.043877 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0812 06:58:09.045275 (+ 1398us) server_negotiation.cc:299] Negotiation successful
0812 06:58:09.045390 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":354,"thread_start_us":178,"threads_started":1}
I20260812 06:58:09.048403 4992 tablet_service.cc:1467] Processing CreateTablet for tablet b3b2befaa08e40c0aa6b14bcdab22e3a (DEFAULT_TABLE table=test-table [id=d3d57ae9fd6c4747ac1786360fbd751e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 06:58:09.048388 5309 tablet_service.cc:1467] Processing CreateTablet for tablet b3b2befaa08e40c0aa6b14bcdab22e3a (DEFAULT_TABLE table=test-table [id=d3d57ae9fd6c4747ac1786360fbd751e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 06:58:09.048377 5147 tablet_service.cc:1467] Processing CreateTablet for tablet b3b2befaa08e40c0aa6b14bcdab22e3a (DEFAULT_TABLE table=test-table [id=d3d57ae9fd6c4747ac1786360fbd751e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 06:58:09.049655 5147 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b3b2befaa08e40c0aa6b14bcdab22e3a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 06:58:09.049681 5309 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b3b2befaa08e40c0aa6b14bcdab22e3a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 06:58:09.049681 4992 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b3b2befaa08e40c0aa6b14bcdab22e3a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 06:58:09.057274 5405 tablet_bootstrap.cc:492] T b3b2befaa08e40c0aa6b14bcdab22e3a P 666220fa0cdf403dad047d3011fed16f: Bootstrap starting.
I20260812 06:58:09.058454 5404 tablet_bootstrap.cc:492] T b3b2befaa08e40c0aa6b14bcdab22e3a P 25ef252218c24d6cae8e3f5efff644ea: Bootstrap starting.
I20260812 06:58:09.058588 5406 tablet_bootstrap.cc:492] T b3b2befaa08e40c0aa6b14bcdab22e3a P aa4029c643b448ad9e197e92749c8fe6: Bootstrap starting.
I20260812 06:58:09.059942 5405 tablet_bootstrap.cc:654] T b3b2befaa08e40c0aa6b14bcdab22e3a P 666220fa0cdf403dad047d3011fed16f: Neither blocks nor log segments found. Creating new log.
I20260812 06:58:09.060776 5406 tablet_bootstrap.cc:654] T b3b2befaa08e40c0aa6b14bcdab22e3a P aa4029c643b448ad9e197e92749c8fe6: Neither blocks nor log segments found. Creating new log.
I20260812 06:58:09.060839 5404 tablet_bootstrap.cc:654] T b3b2befaa08e40c0aa6b14bcdab22e3a P 25ef252218c24d6cae8e3f5efff644ea: Neither blocks nor log segments found. Creating new log.
I20260812 06:58:09.061424 5405 log.cc:824] T b3b2befaa08e40c0aa6b14bcdab22e3a P 666220fa0cdf403dad047d3011fed16f: Log is configured to *not* fsync() on all Append() calls
I20260812 06:58:09.061650 5406 log.cc:824] T b3b2befaa08e40c0aa6b14bcdab22e3a P aa4029c643b448ad9e197e92749c8fe6: Log is configured to *not* fsync() on all Append() calls
I20260812 06:58:09.061836 5404 log.cc:824] T b3b2befaa08e40c0aa6b14bcdab22e3a P 25ef252218c24d6cae8e3f5efff644ea: Log is configured to *not* fsync() on all Append() calls
I20260812 06:58:09.064208 5406 tablet_bootstrap.cc:492] T b3b2befaa08e40c0aa6b14bcdab22e3a P aa4029c643b448ad9e197e92749c8fe6: No bootstrap required, opened a new log
I20260812 06:58:09.064209 5404 tablet_bootstrap.cc:492] T b3b2befaa08e40c0aa6b14bcdab22e3a P 25ef252218c24d6cae8e3f5efff644ea: No bootstrap required, opened a new log
I20260812 06:58:09.064209 5405 tablet_bootstrap.cc:492] T b3b2befaa08e40c0aa6b14bcdab22e3a P 666220fa0cdf403dad047d3011fed16f: No bootstrap required, opened a new log
I20260812 06:58:09.064446 5406 ts_tablet_manager.cc:1397] T b3b2befaa08e40c0aa6b14bcdab22e3a P aa4029c643b448ad9e197e92749c8fe6: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260812 06:58:09.064447 5404 ts_tablet_manager.cc:1397] T b3b2befaa08e40c0aa6b14bcdab22e3a P 25ef252218c24d6cae8e3f5efff644ea: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.004s
I20260812 06:58:09.064447 5405 ts_tablet_manager.cc:1397] T b3b2befaa08e40c0aa6b14bcdab22e3a P 666220fa0cdf403dad047d3011fed16f: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260812 06:58:09.068037 5406 raft_consensus.cc:348] T b3b2befaa08e40c0aa6b14bcdab22e3a P aa4029c643b448ad9e197e92749c8fe6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "25ef252218c24d6cae8e3f5efff644ea" member_type: VOTER last_known_addr { host: "127.19.52.66" port: 34423 } } peers { permanent_uuid: "aa4029c643b448ad9e197e92749c8fe6" member_type: VOTER last_known_addr { host: "127.19.52.65" port: 41283 } } peers { permanent_uuid: "666220fa0cdf403dad047d3011fed16f" member_type: VOTER last_known_addr { host: "127.19.52.67" port: 35465 } }
I20260812 06:58:09.068423 5406 raft_consensus.cc:374] T b3b2befaa08e40c0aa6b14bcdab22e3a P aa4029c643b448ad9e197e92749c8fe6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 06:58:09.068483 5406 raft_consensus.cc:729] T b3b2befaa08e40c0aa6b14bcdab22e3a P aa4029c643b448ad9e197e92749c8fe6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aa4029c643b448ad9e197e92749c8fe6, State: Initialized, Role: FOLLOWER
I20260812 06:58:09.068809 5404 raft_consensus.cc:348] T b3b2befaa08e40c0aa6b14bcdab22e3a P 25ef252218c24d6cae8e3f5efff644ea [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "25ef252218c24d6cae8e3f5efff644ea" member_type: VOTER last_known_addr { host: "127.19.52.66" port: 34423 } } peers { permanent_uuid: "aa4029c643b448ad9e197e92749c8fe6" member_type: VOTER last_known_addr { host: "127.19.52.65" port: 41283 } } peers { permanent_uuid: "666220fa0cdf403dad047d3011fed16f" member_type: VOTER last_known_addr { host: "127.19.52.67" port: 35465 } }
I20260812 06:58:09.068873 5405 raft_consensus.cc:348] T b3b2befaa08e40c0aa6b14bcdab22e3a P 666220fa0cdf403dad047d3011fed16f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "25ef252218c24d6cae8e3f5efff644ea" member_type: VOTER last_known_addr { host: "127.19.52.66" port: 34423 } } peers { permanent_uuid: "aa4029c643b448ad9e197e92749c8fe6" member_type: VOTER last_known_addr { host: "127.19.52.65" port: 41283 } } peers { permanent_uuid: "666220fa0cdf403dad047d3011fed16f" member_type: VOTER last_known_addr { host: "127.19.52.67" port: 35465 } }
I20260812 06:58:09.069103 5405 raft_consensus.cc:374] T b3b2befaa08e40c0aa6b14bcdab22e3a P 666220fa0cdf403dad047d3011fed16f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 06:58:09.069118 5404 raft_consensus.cc:374] T b3b2befaa08e40c0aa6b14bcdab22e3a P 25ef252218c24d6cae8e3f5efff644ea [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 06:58:09.069010 5406 consensus_queue.cc:260] T b3b2befaa08e40c0aa6b14bcdab22e3a P aa4029c643b448ad9e197e92749c8fe6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "25ef252218c24d6cae8e3f5efff644ea" member_type: VOTER last_known_addr { host: "127.19.52.66" port: 34423 } } peers { permanent_uuid: "aa4029c643b448ad9e197e92749c8fe6" member_type: VOTER last_known_addr { host: "127.19.52.65" port: 41283 } } peers { permanent_uuid: "666220fa0cdf403dad047d3011fed16f" member_type: VOTER last_known_addr { host: "127.19.52.67" port: 35465 } }
I20260812 06:58:09.069161 5405 raft_consensus.cc:729] T b3b2befaa08e40c0aa6b14bcdab22e3a P 666220fa0cdf403dad047d3011fed16f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 666220fa0cdf403dad047d3011fed16f, State: Initialized, Role: FOLLOWER
I20260812 06:58:09.069169 5404 raft_consensus.cc:729] T b3b2befaa08e40c0aa6b14bcdab22e3a P 25ef252218c24d6cae8e3f5efff644ea [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 25ef252218c24d6cae8e3f5efff644ea, State: Initialized, Role: FOLLOWER
I20260812 06:58:09.069751 5405 consensus_queue.cc:260] T b3b2befaa08e40c0aa6b14bcdab22e3a P 666220fa0cdf403dad047d3011fed16f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "25ef252218c24d6cae8e3f5efff644ea" member_type: VOTER last_known_addr { host: "127.19.52.66" port: 34423 } } peers { permanent_uuid: "aa4029c643b448ad9e197e92749c8fe6" member_type: VOTER last_known_addr { host: "127.19.52.65" port: 41283 } } peers { permanent_uuid: "666220fa0cdf403dad047d3011fed16f" member_type: VOTER last_known_addr { host: "127.19.52.67" port: 35465 } }
I20260812 06:58:09.069751 5404 consensus_queue.cc:260] T b3b2befaa08e40c0aa6b14bcdab22e3a P 25ef252218c24d6cae8e3f5efff644ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "25ef252218c24d6cae8e3f5efff644ea" member_type: VOTER last_known_addr { host: "127.19.52.66" port: 34423 } } peers { permanent_uuid: "aa4029c643b448ad9e197e92749c8fe6" member_type: VOTER last_known_addr { host: "127.19.52.65" port: 41283 } } peers { permanent_uuid: "666220fa0cdf403dad047d3011fed16f" member_type: VOTER last_known_addr { host: "127.19.52.67" port: 35465 } }
I20260812 06:58:09.070281 5073 heartbeater.cc:503] Master 127.19.52.126:36401 was elected leader, sending a full tablet report...
I20260812 06:58:09.070372 5406 ts_tablet_manager.cc:1428] T b3b2befaa08e40c0aa6b14bcdab22e3a P aa4029c643b448ad9e197e92749c8fe6: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260812 06:58:09.070801 5405 ts_tablet_manager.cc:1428] T b3b2befaa08e40c0aa6b14bcdab22e3a P 666220fa0cdf403dad047d3011fed16f: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260812 06:58:09.070801 5404 ts_tablet_manager.cc:1428] T b3b2befaa08e40c0aa6b14bcdab22e3a P 25ef252218c24d6cae8e3f5efff644ea: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260812 06:58:09.071179 5227 heartbeater.cc:503] Master 127.19.52.126:36401 was elected leader, sending a full tablet report...
I20260812 06:58:09.071568 5385 heartbeater.cc:503] Master 127.19.52.126:36401 was elected leader, sending a full tablet report...
I20260812 06:58:09.077960 5412 raft_consensus.cc:482] T b3b2befaa08e40c0aa6b14bcdab22e3a P 666220fa0cdf403dad047d3011fed16f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 06:58:09.078193 5412 raft_consensus.cc:504] T b3b2befaa08e40c0aa6b14bcdab22e3a P 666220fa0cdf403dad047d3011fed16f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "25ef252218c24d6cae8e3f5efff644ea" member_type: VOTER last_known_addr { host: "127.19.52.66" port: 34423 } } peers { permanent_uuid: "aa4029c643b448ad9e197e92749c8fe6" member_type: VOTER last_known_addr { host: "127.19.52.65" port: 41283 } } peers { permanent_uuid: "666220fa0cdf403dad047d3011fed16f" member_type: VOTER last_known_addr { host: "127.19.52.67" port: 35465 } }
I20260812 06:58:09.079397 5412 leader_election.cc:288] T b3b2befaa08e40c0aa6b14bcdab22e3a P 666220fa0cdf403dad047d3011fed16f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 25ef252218c24d6cae8e3f5efff644ea (127.19.52.66:34423), aa4029c643b448ad9e197e92749c8fe6 (127.19.52.65:41283)
I20260812 06:58:09.085868 5403 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:58:09.079801 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.52.67:56977 (local address 127.19.52.65:41283)
0812 06:58:09.080200 (+ 399us) server_negotiation.cc:206] Beginning negotiation
0812 06:58:09.080206 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 06:58:09.080749 (+ 543us) server_negotiation.cc:407] Connection header received
0812 06:58:09.080877 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:58:09.080880 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:58:09.080935 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:58:09.081065 (+ 130us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 06:58:09.081488 (+ 423us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:58:09.082313 (+ 825us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:58:09.084241 (+ 1928us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:58:09.084918 (+ 677us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:58:09.085059 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:58:09.085116 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0812 06:58:09.085625 (+ 509us) server_negotiation.cc:299] Negotiation successful
0812 06:58:09.085723 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260812 06:58:09.085868 5399 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:58:09.079817 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.52.67:48235 (local address 127.19.52.66:34423)
0812 06:58:09.080220 (+ 403us) server_negotiation.cc:206] Beginning negotiation
0812 06:58:09.080227 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 06:58:09.080821 (+ 594us) server_negotiation.cc:407] Connection header received
0812 06:58:09.080899 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:58:09.080902 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:58:09.080950 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:58:09.081046 (+ 96us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 06:58:09.081488 (+ 442us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:58:09.082313 (+ 825us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:58:09.084241 (+ 1928us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:58:09.084916 (+ 675us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:58:09.085059 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:58:09.085116 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0812 06:58:09.085625 (+ 509us) server_negotiation.cc:299] Negotiation successful
0812 06:58:09.085723 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260812 06:58:09.085901 5388 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:58:09.079650 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.65:41283 (local address 127.19.52.67:56977)
0812 06:58:09.080628 (+ 978us) negotiation.cc:107] Waiting for socket to connect
0812 06:58:09.080647 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0812 06:58:09.080771 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:58:09.081074 (+ 303us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:58:09.081076 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:58:09.081119 (+ 43us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 06:58:09.081330 (+ 211us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:58:09.081337 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:58:09.082476 (+ 1139us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:58:09.082481 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:58:09.084084 (+ 1603us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:58:09.084098 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:58:09.085136 (+ 1038us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:58:09.085143 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:58:09.085344 (+ 201us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:58:09.085413 (+ 69us) client_negotiation.cc:769] Sending connection context
0812 06:58:09.085576 (+ 163us) client_negotiation.cc:240] Negotiation successful
0812 06:58:09.085707 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":604,"mutex_wait_us":285,"spinlock_wait_cycles":8448}
I20260812 06:58:09.085923 5413 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:58:09.079657 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.66:34423 (local address 127.19.52.67:48235)
0812 06:58:09.080714 (+ 1057us) negotiation.cc:107] Waiting for socket to connect
0812 06:58:09.080727 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 06:58:09.080816 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:58:09.081074 (+ 258us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:58:09.081076 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:58:09.081117 (+ 41us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 06:58:09.081330 (+ 213us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:58:09.081337 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:58:09.082494 (+ 1157us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:58:09.082497 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:58:09.084083 (+ 1586us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:58:09.084098 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:58:09.085175 (+ 1077us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:58:09.085178 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:58:09.085348 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:58:09.085410 (+ 62us) client_negotiation.cc:769] Sending connection context
0812 06:58:09.085576 (+ 166us) client_negotiation.cc:240] Negotiation successful
0812 06:58:09.085707 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":981,"mutex_wait_us":261,"spinlock_wait_cycles":28544,"thread_start_us":143,"threads_started":1}
I20260812 06:58:09.086740 5015 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b3b2befaa08e40c0aa6b14bcdab22e3a" candidate_uuid: "666220fa0cdf403dad047d3011fed16f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aa4029c643b448ad9e197e92749c8fe6" is_pre_election: true
I20260812 06:58:09.086822 5172 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b3b2befaa08e40c0aa6b14bcdab22e3a" candidate_uuid: "666220fa0cdf403dad047d3011fed16f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "25ef252218c24d6cae8e3f5efff644ea" is_pre_election: true
I20260812 06:58:09.087114 5015 raft_consensus.cc:2454] T b3b2befaa08e40c0aa6b14bcdab22e3a P aa4029c643b448ad9e197e92749c8fe6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 666220fa0cdf403dad047d3011fed16f in term 0.
I20260812 06:58:09.087142 5172 raft_consensus.cc:2454] T b3b2befaa08e40c0aa6b14bcdab22e3a P 25ef252218c24d6cae8e3f5efff644ea [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 666220fa0cdf403dad047d3011fed16f in term 0.
I20260812 06:58:09.087847 5256 leader_election.cc:302] T b3b2befaa08e40c0aa6b14bcdab22e3a P 666220fa0cdf403dad047d3011fed16f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 666220fa0cdf403dad047d3011fed16f, aa4029c643b448ad9e197e92749c8fe6; no voters:
I20260812 06:58:09.088241 5412 raft_consensus.cc:2781] T b3b2befaa08e40c0aa6b14bcdab22e3a P 666220fa0cdf403dad047d3011fed16f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 06:58:09.088604 5412 raft_consensus.cc:482] T b3b2befaa08e40c0aa6b14bcdab22e3a P 666220fa0cdf403dad047d3011fed16f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 06:58:09.088699 5412 raft_consensus.cc:3037] T b3b2befaa08e40c0aa6b14bcdab22e3a P 666220fa0cdf403dad047d3011fed16f [term 0 FOLLOWER]: Advancing to term 1
I20260812 06:58:09.089811 5412 raft_consensus.cc:504] T b3b2befaa08e40c0aa6b14bcdab22e3a P 666220fa0cdf403dad047d3011fed16f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "25ef252218c24d6cae8e3f5efff644ea" member_type: VOTER last_known_addr { host: "127.19.52.66" port: 34423 } } peers { permanent_uuid: "aa4029c643b448ad9e197e92749c8fe6" member_type: VOTER last_known_addr { host: "127.19.52.65" port: 41283 } } peers { permanent_uuid: "666220fa0cdf403dad047d3011fed16f" member_type: VOTER last_known_addr { host: "127.19.52.67" port: 35465 } }
I20260812 06:58:09.090193 5412 leader_election.cc:288] T b3b2befaa08e40c0aa6b14bcdab22e3a P 666220fa0cdf403dad047d3011fed16f [CANDIDATE]: Term 1 election: Requested vote from peers 25ef252218c24d6cae8e3f5efff644ea (127.19.52.66:34423), aa4029c643b448ad9e197e92749c8fe6 (127.19.52.65:41283)
I20260812 06:58:09.090663 5172 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b3b2befaa08e40c0aa6b14bcdab22e3a" candidate_uuid: "666220fa0cdf403dad047d3011fed16f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "25ef252218c24d6cae8e3f5efff644ea"
I20260812 06:58:09.090667 5015 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b3b2befaa08e40c0aa6b14bcdab22e3a" candidate_uuid: "666220fa0cdf403dad047d3011fed16f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aa4029c643b448ad9e197e92749c8fe6"
I20260812 06:58:09.090821 5172 raft_consensus.cc:3037] T b3b2befaa08e40c0aa6b14bcdab22e3a P 25ef252218c24d6cae8e3f5efff644ea [term 0 FOLLOWER]: Advancing to term 1
I20260812 06:58:09.090832 5015 raft_consensus.cc:3037] T b3b2befaa08e40c0aa6b14bcdab22e3a P aa4029c643b448ad9e197e92749c8fe6 [term 0 FOLLOWER]: Advancing to term 1
I20260812 06:58:09.091933 5015 raft_consensus.cc:2454] T b3b2befaa08e40c0aa6b14bcdab22e3a P aa4029c643b448ad9e197e92749c8fe6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 666220fa0cdf403dad047d3011fed16f in term 1.
I20260812 06:58:09.091936 5172 raft_consensus.cc:2454] T b3b2befaa08e40c0aa6b14bcdab22e3a P 25ef252218c24d6cae8e3f5efff644ea [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 666220fa0cdf403dad047d3011fed16f in term 1.
I20260812 06:58:09.092377 5255 leader_election.cc:302] T b3b2befaa08e40c0aa6b14bcdab22e3a P 666220fa0cdf403dad047d3011fed16f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 25ef252218c24d6cae8e3f5efff644ea, 666220fa0cdf403dad047d3011fed16f, aa4029c643b448ad9e197e92749c8fe6; no voters:
I20260812 06:58:09.092710 5412 raft_consensus.cc:2781] T b3b2befaa08e40c0aa6b14bcdab22e3a P 666220fa0cdf403dad047d3011fed16f [term 1 FOLLOWER]: Leader election won for term 1
I20260812 06:58:09.093052 5412 raft_consensus.cc:686] T b3b2befaa08e40c0aa6b14bcdab22e3a P 666220fa0cdf403dad047d3011fed16f [term 1 LEADER]: Becoming Leader. State: Replica: 666220fa0cdf403dad047d3011fed16f, State: Running, Role: LEADER
I20260812 06:58:09.093425 5412 consensus_queue.cc:237] T b3b2befaa08e40c0aa6b14bcdab22e3a P 666220fa0cdf403dad047d3011fed16f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "25ef252218c24d6cae8e3f5efff644ea" member_type: VOTER last_known_addr { host: "127.19.52.66" port: 34423 } } peers { permanent_uuid: "aa4029c643b448ad9e197e92749c8fe6" member_type: VOTER last_known_addr { host: "127.19.52.65" port: 41283 } } peers { permanent_uuid: "666220fa0cdf403dad047d3011fed16f" member_type: VOTER last_known_addr { host: "127.19.52.67" port: 35465 } }
I20260812 06:58:09.097114 4846 catalog_manager.cc:5495] T b3b2befaa08e40c0aa6b14bcdab22e3a P 666220fa0cdf403dad047d3011fed16f reported cstate change: term changed from 0 to 1, leader changed from <none> to 666220fa0cdf403dad047d3011fed16f (127.19.52.67). New cstate: current_term: 1 leader_uuid: "666220fa0cdf403dad047d3011fed16f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "25ef252218c24d6cae8e3f5efff644ea" member_type: VOTER last_known_addr { host: "127.19.52.66" port: 34423 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "aa4029c643b448ad9e197e92749c8fe6" member_type: VOTER last_known_addr { host: "127.19.52.65" port: 41283 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "666220fa0cdf403dad047d3011fed16f" member_type: VOTER last_known_addr { host: "127.19.52.67" port: 35465 } health_report { overall_health: HEALTHY } } }
I20260812 06:58:09.112746 5402 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:58:09.107357 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44482 (local address 127.19.52.67:35465)
0812 06:58:09.107549 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0812 06:58:09.107554 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 06:58:09.107571 (+ 17us) server_negotiation.cc:407] Connection header received
0812 06:58:09.107663 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:58:09.107665 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:58:09.107723 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:58:09.107816 (+ 93us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 06:58:09.108310 (+ 494us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:58:09.109306 (+ 996us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:58:09.110454 (+ 1148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:58:09.111034 (+ 580us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:58:09.111145 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:58:09.111565 (+ 420us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 06:58:09.112082 (+ 517us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 06:58:09.112188 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0812 06:58:09.112353 (+ 165us) server_negotiation.cc:299] Negotiation successful
0812 06:58:09.112536 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260812 06:58:09.119433 5015 raft_consensus.cc:1261] T b3b2befaa08e40c0aa6b14bcdab22e3a P aa4029c643b448ad9e197e92749c8fe6 [term 1 FOLLOWER]: Refusing update from remote peer 666220fa0cdf403dad047d3011fed16f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 06:58:09.119433 5172 raft_consensus.cc:1261] T b3b2befaa08e40c0aa6b14bcdab22e3a P 25ef252218c24d6cae8e3f5efff644ea [term 1 FOLLOWER]: Refusing update from remote peer 666220fa0cdf403dad047d3011fed16f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 06:58:09.120371 5412 consensus_queue.cc:1035] T b3b2befaa08e40c0aa6b14bcdab22e3a P 666220fa0cdf403dad047d3011fed16f [LEADER]: Connected to new peer: Peer: permanent_uuid: "aa4029c643b448ad9e197e92749c8fe6" member_type: VOTER last_known_addr { host: "127.19.52.65" port: 41283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 06:58:09.120661 5414 consensus_queue.cc:1035] T b3b2befaa08e40c0aa6b14bcdab22e3a P 666220fa0cdf403dad047d3011fed16f [LEADER]: Connected to new peer: Peer: permanent_uuid: "25ef252218c24d6cae8e3f5efff644ea" member_type: VOTER last_known_addr { host: "127.19.52.66" port: 34423 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 06:58:09.128631 5417 mvcc.cc:205] Tried to move back new op lower bound from 7317577273824366592 to 7317577273729495040. Current Snapshot: MvccSnapshot[applied={T|T < 7317577273824366592}]
I20260812 06:58:09.137586 5420 mvcc.cc:205] Tried to move back new op lower bound from 7317577273824366592 to 7317577273729495040. Current Snapshot: MvccSnapshot[applied={T|T < 7317577273824366592}]
I20260812 06:58:09.140667 4843 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56264:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 06:58:09.140868 4843 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56264:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 06:58:09.143744 4843 catalog_manager.cc:5782] T 00000000000000000000000000000000 P f4608298ae804b898d7b5b8077543057: Sending DeleteTablet for 3 replicas of tablet b3b2befaa08e40c0aa6b14bcdab22e3a
I20260812 06:58:09.144657 4992 tablet_service.cc:1514] Processing DeleteTablet for tablet b3b2befaa08e40c0aa6b14bcdab22e3a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 06:58:09 UTC) from {username='kudu', principal='kudu/127.19.52.126@KRBTEST.COM'} at 127.0.0.1:51582
I20260812 06:58:09.144658 5147 tablet_service.cc:1514] Processing DeleteTablet for tablet b3b2befaa08e40c0aa6b14bcdab22e3a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 06:58:09 UTC) from {username='kudu', principal='kudu/127.19.52.126@KRBTEST.COM'} at 127.0.0.1:42240
I20260812 06:58:09.145262 5427 tablet_replica.cc:330] T b3b2befaa08e40c0aa6b14bcdab22e3a P aa4029c643b448ad9e197e92749c8fe6: stopping tablet replica
I20260812 06:58:09.145687 5427 raft_consensus.cc:2229] T b3b2befaa08e40c0aa6b14bcdab22e3a P aa4029c643b448ad9e197e92749c8fe6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 06:58:09.145721 19665 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu with pid 4920
I20260812 06:58:09.146176 5427 raft_consensus.cc:2258] T b3b2befaa08e40c0aa6b14bcdab22e3a P aa4029c643b448ad9e197e92749c8fe6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 06:58:09.146433 5309 tablet_service.cc:1514] Processing DeleteTablet for tablet b3b2befaa08e40c0aa6b14bcdab22e3a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 06:58:09 UTC) from {username='kudu', principal='kudu/127.19.52.126@KRBTEST.COM'} at 127.0.0.1:44480
I20260812 06:58:09.146509 5426 tablet_replica.cc:330] T b3b2befaa08e40c0aa6b14bcdab22e3a P 25ef252218c24d6cae8e3f5efff644ea: stopping tablet replica
I20260812 06:58:09.146961 5426 raft_consensus.cc:2229] T b3b2befaa08e40c0aa6b14bcdab22e3a P 25ef252218c24d6cae8e3f5efff644ea [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 06:58:09.147351 5426 raft_consensus.cc:2258] T b3b2befaa08e40c0aa6b14bcdab22e3a P 25ef252218c24d6cae8e3f5efff644ea [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 06:58:09.149658 5426 ts_tablet_manager.cc:1905] T b3b2befaa08e40c0aa6b14bcdab22e3a P 25ef252218c24d6cae8e3f5efff644ea: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 06:58:09.150246 5428 tablet_replica.cc:330] T b3b2befaa08e40c0aa6b14bcdab22e3a P 666220fa0cdf403dad047d3011fed16f: stopping tablet replica
I20260812 06:58:09.150707 5428 raft_consensus.cc:2229] T b3b2befaa08e40c0aa6b14bcdab22e3a P 666220fa0cdf403dad047d3011fed16f [term 1 LEADER]: Raft consensus shutting down.
I20260812 06:58:09.151309 5428 raft_consensus.cc:2258] T b3b2befaa08e40c0aa6b14bcdab22e3a P 666220fa0cdf403dad047d3011fed16f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 06:58:09.153373 5426 ts_tablet_manager.cc:1918] T b3b2befaa08e40c0aa6b14bcdab22e3a P 25ef252218c24d6cae8e3f5efff644ea: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 06:58:09.153548 5426 log.cc:1196] T b3b2befaa08e40c0aa6b14bcdab22e3a P 25ef252218c24d6cae8e3f5efff644ea: Deleting WAL directory at /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517687384214-19665-0/minicluster-data/ts-1/wal/wals/b3b2befaa08e40c0aa6b14bcdab22e3a
I20260812 06:58:09.154004 5426 ts_tablet_manager.cc:1939] T b3b2befaa08e40c0aa6b14bcdab22e3a P 25ef252218c24d6cae8e3f5efff644ea: Deleting consensus metadata
I20260812 06:58:09.154232 5428 ts_tablet_manager.cc:1905] T b3b2befaa08e40c0aa6b14bcdab22e3a P 666220fa0cdf403dad047d3011fed16f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 06:58:09.155579 4830 catalog_manager.cc:4841] TS 25ef252218c24d6cae8e3f5efff644ea (127.19.52.66:34423): tablet b3b2befaa08e40c0aa6b14bcdab22e3a (table test-table [id=d3d57ae9fd6c4747ac1786360fbd751e]) successfully deleted
I20260812 06:58:09.158990 5428 ts_tablet_manager.cc:1918] T b3b2befaa08e40c0aa6b14bcdab22e3a P 666220fa0cdf403dad047d3011fed16f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 06:58:09.159229 5428 log.cc:1196] T b3b2befaa08e40c0aa6b14bcdab22e3a P 666220fa0cdf403dad047d3011fed16f: Deleting WAL directory at /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517687384214-19665-0/minicluster-data/ts-2/wal/wals/b3b2befaa08e40c0aa6b14bcdab22e3a
I20260812 06:58:09.159731 5428 ts_tablet_manager.cc:1939] T b3b2befaa08e40c0aa6b14bcdab22e3a P 666220fa0cdf403dad047d3011fed16f: Deleting consensus metadata
I20260812 06:58:09.162093 4829 catalog_manager.cc:4841] TS 666220fa0cdf403dad047d3011fed16f (127.19.52.67:35465): tablet b3b2befaa08e40c0aa6b14bcdab22e3a (table test-table [id=d3d57ae9fd6c4747ac1786360fbd751e]) successfully deleted
W20260812 06:58:09.169730 4831 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.19.52.65:41283 (error 108)
I20260812 06:58:09.170480 19665 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu with pid 5079
I20260812 06:58:09.173019 5400 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:58:09.170725 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.65:41283 (local address 127.0.0.1:51598)
0812 06:58:09.172810 (+ 2085us) negotiation.cc:107] Waiting for socket to connect
0812 06:58:09.172912 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.52.65:41283: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1791}
W20260812 06:58:09.173275 4831 catalog_manager.cc:4584] TS aa4029c643b448ad9e197e92749c8fe6 (127.19.52.65:41283): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet b3b2befaa08e40c0aa6b14bcdab22e3a: Network error: Client connection negotiation failed: client connection to 127.19.52.65:41283: connect: Connection refused (error 111)
I20260812 06:58:09.180321 19665 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu with pid 5233
I20260812 06:58:09.189881 19665 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu with pid 4811
I20260812 06:58:29.200004 19665 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu
/tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517687384214-19665-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517687384214-19665-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517687384214-19665-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517687384214-19665-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.52.126:36401
--webserver_interface=127.19.52.126
--webserver_port=40333
--keytab_file=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517687384214-19665-0/krb5kdc/kudu_127.19.52.126.keytab
--principal=kudu/127.19.52.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.52.84:35215
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.19.52.126:36401
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--tsk_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517687384214-19665-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517687384214-19665-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517687384214-19665-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 06:58:29.322192 5430 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:58:29.322503 5430 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:58:29.322548 5430 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 06:58:29.326632 5430 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260812 06:58:29.326713 5430 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260812 06:58:29.326730 5430 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260812 06:58:29.326743 5430 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 06:58:29.326763 5430 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 06:58:29.326781 5430 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 06:58:29.326798 5430 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 06:58:29.326813 5430 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 06:58:29.332643 5430 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.52.84:35215
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517687384214-19665-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517687384214-19665-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_private_key_password_cmd=echo foo
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.19.52.126:36401
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517687384214-19665-0/krb5kdc/kudu_127.19.52.126.keytab
--principal=kudu/127.19.52.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.52.126:36401
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517687384214-19665-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.19.52.126
--webserver_port=40333
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5430
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517687384214-19665-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 06:15:54 UTC on ccycloud.quasar-rnfzce.root.comops.site
build id 4581989
I20260812 06:58:29.333905 5430 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:58:29.335098 5430 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 06:58:29.351182 5438 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:58:29.351229 5440 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:58:29.351261 5437 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:58:29.351603 5430 server_base.cc:1034] running on GCE node
I20260812 06:58:29.352320 5430 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:58:29.353713 5430 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:58:29.354933 5430 hybrid_clock.cc:648] HybridClock initialized: now 1786517909354915 us; error 33 us; skew 500 ppm
Aug 12 06:58:29 dist-test-slave-65mx krb5kdc[4794](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517909, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.52.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 06:58:29.363051 5430 init.cc:377] Logged in from keytab as kudu/127.19.52.126@KRBTEST.COM (short username kudu)
I20260812 06:58:29.364262 5430 webserver.cc:460] Webserver started at http://127.19.52.126:40333/ using document root <none> and password file <none>
I20260812 06:58:29.365011 5430 fs_manager.cc:359] Metadata directory not provided
I20260812 06:58:29.365152 5430 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:58:29.369707 5430 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260812 06:58:29.372335 5451 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 06:58:29.373958 5430 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 06:58:29.374155 5430 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517687384214-19665-0/minicluster-data/master-0/data,/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517687384214-19665-0/minicluster-data/master-0/wal
uuid: "f4608298ae804b898d7b5b8077543057"
format_stamp: "Formatted at 2026-08-12 06:58:08 on dist-test-slave-65mx"
server_key: "e1431a0315a64cc0d3a002aa1a879aa3"
server_key_iv: "05eea69e240961fe06492fbfcc00dbef"
server_key_version: "encryptionkey@0"
I20260812 06:58:29.374851 5430 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517687384214-19665-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517687384214-19665-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517687384214-19665-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:58:29.389199 5430 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 06:58:29.392828 5430 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 06:58:29.393066 5430 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:58:29.401217 5430 rpc_server.cc:307] RPC server started. Bound to: 127.19.52.126:36401
I20260812 06:58:29.401567 5504 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.52.126:36401 every 8 connection(s)
I20260812 06:58:29.402424 5430 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517687384214-19665-0/minicluster-data/master-0/data/info.pb
I20260812 06:58:29.407235 5506 sys_catalog.cc:263] Verifying existing consensus state
I20260812 06:58:29.409555 5506 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f4608298ae804b898d7b5b8077543057: Bootstrap starting.
I20260812 06:58:29.410871 19665 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu as pid 5430
I20260812 06:58:29.411449 19665 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu
/tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517687384214-19665-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517687384214-19665-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517687384214-19665-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517687384214-19665-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.52.65:41283
--local_ip_for_outbound_sockets=127.19.52.65
--tserver_master_addrs=127.19.52.126:36401
--webserver_port=44717
--webserver_interface=127.19.52.65
--keytab_file=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517687384214-19665-0/krb5kdc/kudu_127.19.52.65.keytab
--principal=kudu/127.19.52.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.52.84:35215
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517687384214-19665-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517687384214-19665-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517687384214-19665-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 06:58:29.433010 5506 log.cc:824] T 00000000000000000000000000000000 P f4608298ae804b898d7b5b8077543057: Log is configured to *not* fsync() on all Append() calls
I20260812 06:58:29.442209 5506 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f4608298ae804b898d7b5b8077543057: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260812 06:58:29.442672 5506 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f4608298ae804b898d7b5b8077543057: Bootstrap complete.
I20260812 06:58:29.447062 5506 raft_consensus.cc:348] T 00000000000000000000000000000000 P f4608298ae804b898d7b5b8077543057 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f4608298ae804b898d7b5b8077543057" member_type: VOTER last_known_addr { host: "127.19.52.126" port: 36401 } }
I20260812 06:58:29.447499 5506 raft_consensus.cc:729] T 00000000000000000000000000000000 P f4608298ae804b898d7b5b8077543057 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: f4608298ae804b898d7b5b8077543057, State: Initialized, Role: FOLLOWER
I20260812 06:58:29.448114 5506 consensus_queue.cc:260] T 00000000000000000000000000000000 P f4608298ae804b898d7b5b8077543057 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f4608298ae804b898d7b5b8077543057" member_type: VOTER last_known_addr { host: "127.19.52.126" port: 36401 } }
I20260812 06:58:29.448278 5506 raft_consensus.cc:388] T 00000000000000000000000000000000 P f4608298ae804b898d7b5b8077543057 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 06:58:29.448411 5506 raft_consensus.cc:482] T 00000000000000000000000000000000 P f4608298ae804b898d7b5b8077543057 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 06:58:29.448732 5506 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f4608298ae804b898d7b5b8077543057 [term 1 FOLLOWER]: Advancing to term 2
I20260812 06:58:29.450968 5506 raft_consensus.cc:504] T 00000000000000000000000000000000 P f4608298ae804b898d7b5b8077543057 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f4608298ae804b898d7b5b8077543057" member_type: VOTER last_known_addr { host: "127.19.52.126" port: 36401 } }
I20260812 06:58:29.451383 5506 leader_election.cc:302] T 00000000000000000000000000000000 P f4608298ae804b898d7b5b8077543057 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f4608298ae804b898d7b5b8077543057; no voters:
I20260812 06:58:29.451766 5506 leader_election.cc:288] T 00000000000000000000000000000000 P f4608298ae804b898d7b5b8077543057 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260812 06:58:29.451869 5510 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f4608298ae804b898d7b5b8077543057 [term 2 FOLLOWER]: Leader election won for term 2
I20260812 06:58:29.452059 5510 raft_consensus.cc:686] T 00000000000000000000000000000000 P f4608298ae804b898d7b5b8077543057 [term 2 LEADER]: Becoming Leader. State: Replica: f4608298ae804b898d7b5b8077543057, State: Running, Role: LEADER
I20260812 06:58:29.452374 5510 consensus_queue.cc:237] T 00000000000000000000000000000000 P f4608298ae804b898d7b5b8077543057 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f4608298ae804b898d7b5b8077543057" member_type: VOTER last_known_addr { host: "127.19.52.126" port: 36401 } }
I20260812 06:58:29.453195 5506 sys_catalog.cc:564] T 00000000000000000000000000000000 P f4608298ae804b898d7b5b8077543057 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 06:58:29.453459 5510 sys_catalog.cc:455] T 00000000000000000000000000000000 P f4608298ae804b898d7b5b8077543057 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f4608298ae804b898d7b5b8077543057. Latest consensus state: current_term: 2 leader_uuid: "f4608298ae804b898d7b5b8077543057" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f4608298ae804b898d7b5b8077543057" member_type: VOTER last_known_addr { host: "127.19.52.126" port: 36401 } } }
I20260812 06:58:29.453593 5510 sys_catalog.cc:458] T 00000000000000000000000000000000 P f4608298ae804b898d7b5b8077543057 [sys.catalog]: This master's current role is: LEADER
I20260812 06:58:29.454034 5516 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 06:58:29.454013 5513 sys_catalog.cc:455] T 00000000000000000000000000000000 P f4608298ae804b898d7b5b8077543057 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "f4608298ae804b898d7b5b8077543057" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f4608298ae804b898d7b5b8077543057" member_type: VOTER last_known_addr { host: "127.19.52.126" port: 36401 } } }
I20260812 06:58:29.455950 5513 sys_catalog.cc:458] T 00000000000000000000000000000000 P f4608298ae804b898d7b5b8077543057 [sys.catalog]: This master's current role is: LEADER
I20260812 06:58:29.459445 5516 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 06:58:29.460368 5516 catalog_manager.cc:1259] Loaded cluster ID: 6dd4eb238b9f4adabe01ec004ec213af
I20260812 06:58:29.460436 5516 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 06:58:29.462975 5516 catalog_manager.cc:1504] Loading token signing keys...
I20260812 06:58:29.463739 5516 catalog_manager.cc:5879] T 00000000000000000000000000000000 P f4608298ae804b898d7b5b8077543057: Loaded TSK: 0
I20260812 06:58:29.480321 5516 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f4608298ae804b898d7b5b8077543057: Generated new TSK 1
I20260812 06:58:29.480690 5516 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260812 06:58:29.544266 5508 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:58:29.544565 5508 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:58:29.544668 5508 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 06:58:29.549220 5508 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 06:58:29.549292 5508 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 06:58:29.549355 5508 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.52.65
I20260812 06:58:29.555241 5508 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.52.84:35215
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517687384214-19665-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517687384214-19665-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517687384214-19665-0/krb5kdc/kudu_127.19.52.65.keytab
--principal=kudu/127.19.52.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.52.65:41283
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517687384214-19665-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.19.52.65
--webserver_port=44717
--tserver_master_addrs=127.19.52.126:36401
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5508
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.52.65
--log_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517687384214-19665-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 06:15:54 UTC on ccycloud.quasar-rnfzce.root.comops.site
build id 4581989
I20260812 06:58:29.556380 5508 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:58:29.557498 5508 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 06:58:29.567953 5538 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:58:29.568138 5508 server_base.cc:1034] running on GCE node
W20260812 06:58:29.567934 5543 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:58:29.567991 5539 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:58:29.568866 5508 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:58:29.569612 5508 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:58:29.570829 5508 hybrid_clock.cc:648] HybridClock initialized: now 1786517909570790 us; error 57 us; skew 500 ppm
Aug 12 06:58:29 dist-test-slave-65mx krb5kdc[4794](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517909, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.52.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 06:58:29.574195 5508 init.cc:377] Logged in from keytab as kudu/127.19.52.65@KRBTEST.COM (short username kudu)
I20260812 06:58:29.575304 5508 webserver.cc:460] Webserver started at http://127.19.52.65:44717/ using document root <none> and password file <none>
I20260812 06:58:29.575908 5508 fs_manager.cc:359] Metadata directory not provided
I20260812 06:58:29.576041 5508 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:58:29.580718 5508 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260812 06:58:29.583204 5552 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 06:58:29.584685 5508 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 06:58:29.584820 5508 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517687384214-19665-0/minicluster-data/ts-0/data,/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517687384214-19665-0/minicluster-data/ts-0/wal
uuid: "aa4029c643b448ad9e197e92749c8fe6"
format_stamp: "Formatted at 2026-08-12 06:58:08 on dist-test-slave-65mx"
server_key: "8850f5c2ac7fdd0cd582238231808ff3"
server_key_iv: "2ac23c925edf745076c39d522dc5cf21"
server_key_version: "encryptionkey@0"
I20260812 06:58:29.585390 5508 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517687384214-19665-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517687384214-19665-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517687384214-19665-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:58:29.608107 5508 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 06:58:29.611948 5508 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 06:58:29.612192 5508 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:58:29.612881 5508 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 06:58:29.614064 5559 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260812 06:58:29.618554 5508 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260812 06:58:29.618641 5508 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.005s user 0.000s sys 0.000s
I20260812 06:58:29.618731 5508 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260812 06:58:29.620528 5508 ts_tablet_manager.cc:610] Registered 1 tablets
I20260812 06:58:29.620664 5508 ts_tablet_manager.cc:589] Time spent register tablets: real 0.002s user 0.000s sys 0.002s
I20260812 06:58:29.620954 5559 tablet_bootstrap.cc:492] T b3b2befaa08e40c0aa6b14bcdab22e3a P aa4029c643b448ad9e197e92749c8fe6: Bootstrap starting.
I20260812 06:58:29.632023 5508 rpc_server.cc:307] RPC server started. Bound to: 127.19.52.65:41283
I20260812 06:58:29.632134 5684 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.52.65:41283 every 8 connection(s)
I20260812 06:58:29.633193 5508 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517687384214-19665-0/minicluster-data/ts-0/data/info.pb
I20260812 06:58:29.634778 19665 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu as pid 5508
I20260812 06:58:29.635296 19665 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu
/tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517687384214-19665-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517687384214-19665-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517687384214-19665-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517687384214-19665-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.52.66:34423
--local_ip_for_outbound_sockets=127.19.52.66
--tserver_master_addrs=127.19.52.126:36401
--webserver_port=37613
--webserver_interface=127.19.52.66
--keytab_file=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517687384214-19665-0/krb5kdc/kudu_127.19.52.66.keytab
--principal=kudu/127.19.52.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.52.84:35215
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517687384214-19665-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517687384214-19665-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517687384214-19665-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 06:58:29.645308 5559 log.cc:824] T b3b2befaa08e40c0aa6b14bcdab22e3a P aa4029c643b448ad9e197e92749c8fe6: Log is configured to *not* fsync() on all Append() calls
Aug 12 06:58:29 dist-test-slave-65mx krb5kdc[4794](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517909, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.52.65@KRBTEST.COM for kudu/127.19.52.126@KRBTEST.COM
I20260812 06:58:29.653985 5688 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:58:29.635693 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.126:36401 (local address 127.19.52.65:35269)
0812 06:58:29.636976 (+ 1283us) negotiation.cc:107] Waiting for socket to connect
0812 06:58:29.637046 (+ 70us) client_negotiation.cc:174] Beginning negotiation
0812 06:58:29.637977 (+ 931us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:58:29.639150 (+ 1173us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:58:29.639159 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:58:29.639634 (+ 475us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:58:29.640107 (+ 473us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:58:29.640122 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:58:29.641547 (+ 1425us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:58:29.641556 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:58:29.642590 (+ 1034us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:58:29.642603 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:58:29.643457 (+ 854us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:58:29.643461 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:58:29.643726 (+ 265us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:58:29.644506 (+ 780us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 06:58:29.644529 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:58:29.646981 (+ 2452us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 06:58:29.650027 (+ 3046us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:58:29.650037 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:58:29.650055 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:58:29.650519 (+ 464us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:58:29.650980 (+ 461us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:58:29.650986 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:58:29.650990 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:58:29.651188 (+ 198us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:58:29.651992 (+ 804us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 06:58:29.651998 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 06:58:29.652303 (+ 305us) client_negotiation.cc:769] Sending connection context
0812 06:58:29.652572 (+ 269us) client_negotiation.cc:240] Negotiation successful
0812 06:58:29.652959 (+ 387us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1009,"thread_start_us":254,"threads_started":1}
I20260812 06:58:29.654031 5687 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:58:29.635709 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.52.65:35269 (local address 127.19.52.126:36401)
0812 06:58:29.636602 (+ 893us) server_negotiation.cc:206] Beginning negotiation
0812 06:58:29.636627 (+ 25us) server_negotiation.cc:399] Waiting for connection header
0812 06:58:29.637214 (+ 587us) server_negotiation.cc:407] Connection header received
0812 06:58:29.638421 (+ 1207us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:58:29.638431 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:58:29.638748 (+ 317us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:58:29.639098 (+ 350us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:58:29.640305 (+ 1207us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:58:29.641383 (+ 1078us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:58:29.642743 (+ 1360us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:58:29.643276 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:58:29.643468 (+ 192us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:58:29.647273 (+ 3805us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:58:29.647281 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:58:29.647298 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:58:29.647337 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:58:29.649847 (+ 2510us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:58:29.650681 (+ 834us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:58:29.650688 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:58:29.650693 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:58:29.650854 (+ 161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:58:29.651322 (+ 468us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:58:29.651326 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:58:29.651328 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:58:29.651846 (+ 518us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:58:29.651998 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0812 06:58:29.652724 (+ 726us) server_negotiation.cc:299] Negotiation successful
0812 06:58:29.653059 (+ 335us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":667,"thread_start_us":130,"threads_started":1}
I20260812 06:58:29.655287 5559 tablet_bootstrap.cc:492] T b3b2befaa08e40c0aa6b14bcdab22e3a P aa4029c643b448ad9e197e92749c8fe6: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260812 06:58:29.655742 5559 tablet_bootstrap.cc:492] T b3b2befaa08e40c0aa6b14bcdab22e3a P aa4029c643b448ad9e197e92749c8fe6: Bootstrap complete.
I20260812 06:58:29.656890 5685 heartbeater.cc:347] Connected to a master server at 127.19.52.126:36401
I20260812 06:58:29.657163 5685 heartbeater.cc:464] Registering TS with master...
I20260812 06:58:29.657279 5559 ts_tablet_manager.cc:1397] T b3b2befaa08e40c0aa6b14bcdab22e3a P aa4029c643b448ad9e197e92749c8fe6: Time spent bootstrapping tablet: real 0.037s user 0.022s sys 0.010s
I20260812 06:58:29.657791 5685 heartbeater.cc:511] Master 127.19.52.126:36401 requested a full tablet report, sending...
I20260812 06:58:29.660120 5559 raft_consensus.cc:348] T b3b2befaa08e40c0aa6b14bcdab22e3a P aa4029c643b448ad9e197e92749c8fe6 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "25ef252218c24d6cae8e3f5efff644ea" member_type: VOTER last_known_addr { host: "127.19.52.66" port: 34423 } } peers { permanent_uuid: "aa4029c643b448ad9e197e92749c8fe6" member_type: VOTER last_known_addr { host: "127.19.52.65" port: 41283 } } peers { permanent_uuid: "666220fa0cdf403dad047d3011fed16f" member_type: VOTER last_known_addr { host: "127.19.52.67" port: 35465 } }
I20260812 06:58:29.660589 5559 raft_consensus.cc:729] T b3b2befaa08e40c0aa6b14bcdab22e3a P aa4029c643b448ad9e197e92749c8fe6 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: aa4029c643b448ad9e197e92749c8fe6, State: Initialized, Role: FOLLOWER
I20260812 06:58:29.660818 5465 ts_manager.cc:194] Registered new tserver with Master: aa4029c643b448ad9e197e92749c8fe6 (127.19.52.65:41283)
I20260812 06:58:29.661091 5559 consensus_queue.cc:260] T b3b2befaa08e40c0aa6b14bcdab22e3a P aa4029c643b448ad9e197e92749c8fe6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "25ef252218c24d6cae8e3f5efff644ea" member_type: VOTER last_known_addr { host: "127.19.52.66" port: 34423 } } peers { permanent_uuid: "aa4029c643b448ad9e197e92749c8fe6" member_type: VOTER last_known_addr { host: "127.19.52.65" port: 41283 } } peers { permanent_uuid: "666220fa0cdf403dad047d3011fed16f" member_type: VOTER last_known_addr { host: "127.19.52.67" port: 35465 } }
I20260812 06:58:29.662065 5559 ts_tablet_manager.cc:1428] T b3b2befaa08e40c0aa6b14bcdab22e3a P aa4029c643b448ad9e197e92749c8fe6: Time spent starting tablet: real 0.005s user 0.003s sys 0.002s
I20260812 06:58:29.664978 5465 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.52.65@KRBTEST.COM'} at 127.19.52.65:35269
I20260812 06:58:29.667448 5685 heartbeater.cc:503] Master 127.19.52.126:36401 was elected leader, sending a full tablet report...
Aug 12 06:58:29 dist-test-slave-65mx krb5kdc[4794](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517909, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.52.126@KRBTEST.COM for kudu/127.19.52.65@KRBTEST.COM
I20260812 06:58:29.676013 5694 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:58:29.664308 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.65:41283 (local address 127.0.0.1:44584)
0812 06:58:29.664910 (+ 602us) negotiation.cc:107] Waiting for socket to connect
0812 06:58:29.664956 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0812 06:58:29.665117 (+ 161us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:58:29.665644 (+ 527us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:58:29.665649 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:58:29.666098 (+ 449us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:58:29.666241 (+ 143us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:58:29.666250 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:58:29.667631 (+ 1381us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:58:29.667634 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:58:29.668643 (+ 1009us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:58:29.668655 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:58:29.669430 (+ 775us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:58:29.669434 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:58:29.669554 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:58:29.670168 (+ 614us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 06:58:29.670180 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:58:29.672014 (+ 1834us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 06:58:29.674050 (+ 2036us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:58:29.674062 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:58:29.674067 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:58:29.674337 (+ 270us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:58:29.674665 (+ 328us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:58:29.674668 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:58:29.674671 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:58:29.674764 (+ 93us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:58:29.675278 (+ 514us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 06:58:29.675285 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 06:58:29.675423 (+ 138us) client_negotiation.cc:769] Sending connection context
0812 06:58:29.675569 (+ 146us) client_negotiation.cc:240] Negotiation successful
0812 06:58:29.675823 (+ 254us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":419,"thread_start_us":151,"threads_started":1}
I20260812 06:58:29.676013 5695 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:58:29.664367 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44584 (local address 127.19.52.65:41283)
0812 06:58:29.665238 (+ 871us) server_negotiation.cc:206] Beginning negotiation
0812 06:58:29.665253 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0812 06:58:29.665280 (+ 27us) server_negotiation.cc:407] Connection header received
0812 06:58:29.665337 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:58:29.665340 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:58:29.665491 (+ 151us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:58:29.665644 (+ 153us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:58:29.666383 (+ 739us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:58:29.667465 (+ 1082us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:58:29.668792 (+ 1327us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:58:29.669307 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:58:29.669428 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:58:29.672186 (+ 2758us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:58:29.672202 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:58:29.672214 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:58:29.672240 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:58:29.673908 (+ 1668us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:58:29.674473 (+ 565us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:58:29.674478 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:58:29.674480 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:58:29.674563 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:58:29.674858 (+ 295us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:58:29.674863 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:58:29.674865 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:58:29.675152 (+ 287us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:58:29.675300 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0812 06:58:29.675653 (+ 353us) server_negotiation.cc:299] Negotiation successful
0812 06:58:29.675823 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":698,"thread_start_us":103,"threads_started":1}
I20260812 06:58:29.677175 5608 tablet_service.cc:1514] Processing DeleteTablet for tablet b3b2befaa08e40c0aa6b14bcdab22e3a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 06:58:09 UTC) from {username='kudu', principal='kudu/127.19.52.126@KRBTEST.COM'} at 127.0.0.1:44584
I20260812 06:58:29.678143 5696 tablet_replica.cc:330] stopping tablet replica
I20260812 06:58:29.678614 5696 raft_consensus.cc:2229] T b3b2befaa08e40c0aa6b14bcdab22e3a P aa4029c643b448ad9e197e92749c8fe6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 06:58:29.678915 5696 raft_consensus.cc:2258] T b3b2befaa08e40c0aa6b14bcdab22e3a P aa4029c643b448ad9e197e92749c8fe6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 06:58:29.680111 5696 ts_tablet_manager.cc:1905] T b3b2befaa08e40c0aa6b14bcdab22e3a P aa4029c643b448ad9e197e92749c8fe6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 06:58:29.685153 5696 ts_tablet_manager.cc:1918] T b3b2befaa08e40c0aa6b14bcdab22e3a P aa4029c643b448ad9e197e92749c8fe6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 06:58:29.685273 5696 log.cc:1196] T b3b2befaa08e40c0aa6b14bcdab22e3a P aa4029c643b448ad9e197e92749c8fe6: Deleting WAL directory at /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517687384214-19665-0/minicluster-data/ts-0/wal/wals/b3b2befaa08e40c0aa6b14bcdab22e3a
I20260812 06:58:29.685588 5696 ts_tablet_manager.cc:1939] T b3b2befaa08e40c0aa6b14bcdab22e3a P aa4029c643b448ad9e197e92749c8fe6: Deleting consensus metadata
I20260812 06:58:29.687978 5455 catalog_manager.cc:4841] TS aa4029c643b448ad9e197e92749c8fe6 (127.19.52.65:41283): tablet b3b2befaa08e40c0aa6b14bcdab22e3a (table test-table [id=d3d57ae9fd6c4747ac1786360fbd751e]) successfully deleted
W20260812 06:58:29.767426 5689 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:58:29.767735 5689 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:58:29.767781 5689 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 06:58:29.772107 5689 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 06:58:29.772194 5689 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 06:58:29.772291 5689 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.52.66
I20260812 06:58:29.778250 5689 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.52.84:35215
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517687384214-19665-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517687384214-19665-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517687384214-19665-0/krb5kdc/kudu_127.19.52.66.keytab
--principal=kudu/127.19.52.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.52.66:34423
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517687384214-19665-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.19.52.66
--webserver_port=37613
--tserver_master_addrs=127.19.52.126:36401
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5689
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.52.66
--log_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517687384214-19665-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 06:15:54 UTC on ccycloud.quasar-rnfzce.root.comops.site
build id 4581989
I20260812 06:58:29.779443 5689 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:58:29.780654 5689 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 06:58:29.790761 5703 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:58:29.790783 5702 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:58:29.790783 5705 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:58:29.791043 5689 server_base.cc:1034] running on GCE node
I20260812 06:58:29.791563 5689 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:58:29.792308 5689 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:58:29.793512 5689 hybrid_clock.cc:648] HybridClock initialized: now 1786517909793490 us; error 67 us; skew 500 ppm
Aug 12 06:58:29 dist-test-slave-65mx krb5kdc[4794](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517909, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.52.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 06:58:29.796382 5689 init.cc:377] Logged in from keytab as kudu/127.19.52.66@KRBTEST.COM (short username kudu)
I20260812 06:58:29.797411 5689 webserver.cc:460] Webserver started at http://127.19.52.66:37613/ using document root <none> and password file <none>
I20260812 06:58:29.798007 5689 fs_manager.cc:359] Metadata directory not provided
I20260812 06:58:29.798127 5689 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:58:29.802641 5689 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260812 06:58:29.804997 5713 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 06:58:29.806406 5689 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 06:58:29.806563 5689 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517687384214-19665-0/minicluster-data/ts-1/data,/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517687384214-19665-0/minicluster-data/ts-1/wal
uuid: "25ef252218c24d6cae8e3f5efff644ea"
format_stamp: "Formatted at 2026-08-12 06:58:08 on dist-test-slave-65mx"
server_key: "8528faeac61ece0467e7f9d0be88de1a"
server_key_iv: "957fc3fb0af5270c72c4d5ba7c779a04"
server_key_version: "encryptionkey@0"
I20260812 06:58:29.807164 5689 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517687384214-19665-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517687384214-19665-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517687384214-19665-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:58:29.821872 5689 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 06:58:29.827185 5689 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 06:58:29.827369 5689 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:58:29.827932 5689 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 06:58:29.828855 5689 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 06:58:29.828908 5689 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 06:58:29.828950 5689 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 06:58:29.828966 5689 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 06:58:29.838369 5689 rpc_server.cc:307] RPC server started. Bound to: 127.19.52.66:34423
I20260812 06:58:29.838426 5847 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.52.66:34423 every 8 connection(s)
I20260812 06:58:29.839450 5689 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517687384214-19665-0/minicluster-data/ts-1/data/info.pb
I20260812 06:58:29.846284 19665 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu as pid 5689
I20260812 06:58:29.846872 19665 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu
/tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517687384214-19665-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517687384214-19665-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517687384214-19665-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517687384214-19665-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.52.67:35465
--local_ip_for_outbound_sockets=127.19.52.67
--tserver_master_addrs=127.19.52.126:36401
--webserver_port=41963
--webserver_interface=127.19.52.67
--keytab_file=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517687384214-19665-0/krb5kdc/kudu_127.19.52.67.keytab
--principal=kudu/127.19.52.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.52.84:35215
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517687384214-19665-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517687384214-19665-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517687384214-19665-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 12 06:58:29 dist-test-slave-65mx krb5kdc[4794](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517909, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.52.66@KRBTEST.COM for kudu/127.19.52.126@KRBTEST.COM
I20260812 06:58:29.856511 5687 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:58:29.841668 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.52.66:53385 (local address 127.19.52.126:36401)
0812 06:58:29.841844 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0812 06:58:29.841849 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 06:58:29.842767 (+ 918us) server_negotiation.cc:407] Connection header received
0812 06:58:29.844126 (+ 1359us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:58:29.844129 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:58:29.844186 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:58:29.844269 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:58:29.845597 (+ 1328us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:58:29.846173 (+ 576us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:58:29.848266 (+ 2093us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:58:29.848738 (+ 472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:58:29.848843 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:58:29.851828 (+ 2985us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:58:29.851831 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:58:29.851833 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:58:29.851862 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:58:29.854103 (+ 2241us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:58:29.854820 (+ 717us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:58:29.854824 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:58:29.854826 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:58:29.854916 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:58:29.855311 (+ 395us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:58:29.855315 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:58:29.855316 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:58:29.855538 (+ 222us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:58:29.855649 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0812 06:58:29.856186 (+ 537us) server_negotiation.cc:299] Negotiation successful
0812 06:58:29.856342 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260812 06:58:29.857470 5850 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:58:29.842081 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.126:36401 (local address 127.19.52.66:53385)
0812 06:58:29.842591 (+ 510us) negotiation.cc:107] Waiting for socket to connect
0812 06:58:29.842636 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0812 06:58:29.843892 (+ 1256us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:58:29.844442 (+ 550us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:58:29.844449 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:58:29.844915 (+ 466us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:58:29.845412 (+ 497us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:58:29.845426 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:58:29.846345 (+ 919us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:58:29.846352 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:58:29.848072 (+ 1720us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:58:29.848090 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:58:29.848937 (+ 847us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:58:29.848944 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:58:29.849209 (+ 265us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:58:29.849974 (+ 765us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 06:58:29.849992 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:58:29.851613 (+ 1621us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 06:58:29.854305 (+ 2692us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:58:29.854314 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:58:29.854335 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:58:29.854704 (+ 369us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:58:29.855012 (+ 308us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:58:29.855018 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:58:29.855021 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:58:29.855174 (+ 153us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:58:29.855649 (+ 475us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 06:58:29.855657 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 06:58:29.855933 (+ 276us) client_negotiation.cc:769] Sending connection context
0812 06:58:29.856145 (+ 212us) client_negotiation.cc:240] Negotiation successful
0812 06:58:29.856532 (+ 387us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":299,"thread_start_us":124,"threads_started":1}
I20260812 06:58:29.859069 5848 heartbeater.cc:347] Connected to a master server at 127.19.52.126:36401
I20260812 06:58:29.859361 5848 heartbeater.cc:464] Registering TS with master...
I20260812 06:58:29.859917 5848 heartbeater.cc:511] Master 127.19.52.126:36401 requested a full tablet report, sending...
I20260812 06:58:29.861188 5465 ts_manager.cc:194] Registered new tserver with Master: 25ef252218c24d6cae8e3f5efff644ea (127.19.52.66:34423)
I20260812 06:58:29.862118 5465 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.52.66@KRBTEST.COM'} at 127.19.52.66:53385
W20260812 06:58:29.975225 5852 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:58:29.975521 5852 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:58:29.975566 5852 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 06:58:29.980028 5852 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 06:58:29.980110 5852 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 06:58:29.980180 5852 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.52.67
I20260812 06:58:29.985961 5852 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.52.84:35215
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517687384214-19665-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517687384214-19665-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517687384214-19665-0/krb5kdc/kudu_127.19.52.67.keytab
--principal=kudu/127.19.52.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.52.67:35465
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517687384214-19665-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.19.52.67
--webserver_port=41963
--tserver_master_addrs=127.19.52.126:36401
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5852
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.52.67
--log_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517687384214-19665-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 06:15:54 UTC on ccycloud.quasar-rnfzce.root.comops.site
build id 4581989
I20260812 06:58:29.987154 5852 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:58:29.988154 5852 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 06:58:29.998309 5859 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:58:29.998330 5857 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:58:29.998296 5861 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:58:29.998672 5852 server_base.cc:1034] running on GCE node
I20260812 06:58:29.999203 5852 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:58:29.999894 5852 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:58:30.001099 5852 hybrid_clock.cc:648] HybridClock initialized: now 1786517910001078 us; error 30 us; skew 500 ppm
Aug 12 06:58:29 dist-test-slave-65mx krb5kdc[4794](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517909, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.52.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 06:58:30.003969 5852 init.cc:377] Logged in from keytab as kudu/127.19.52.67@KRBTEST.COM (short username kudu)
I20260812 06:58:30.004918 5852 webserver.cc:460] Webserver started at http://127.19.52.67:41963/ using document root <none> and password file <none>
I20260812 06:58:30.005519 5852 fs_manager.cc:359] Metadata directory not provided
I20260812 06:58:30.005597 5852 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:58:30.009490 5852 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260812 06:58:30.011821 5870 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 06:58:30.013309 5852 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 06:58:30.013479 5852 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517687384214-19665-0/minicluster-data/ts-2/data,/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517687384214-19665-0/minicluster-data/ts-2/wal
uuid: "666220fa0cdf403dad047d3011fed16f"
format_stamp: "Formatted at 2026-08-12 06:58:08 on dist-test-slave-65mx"
server_key: "76e41badd775cd19f48c41252a471152"
server_key_iv: "8b7221af01d65e7fcbc265f760f1f591"
server_key_version: "encryptionkey@0"
I20260812 06:58:30.014046 5852 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517687384214-19665-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517687384214-19665-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517687384214-19665-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:58:30.027074 5852 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 06:58:30.030368 5852 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 06:58:30.030596 5852 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:58:30.031257 5852 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 06:58:30.032150 5852 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 06:58:30.032197 5852 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 06:58:30.032238 5852 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 06:58:30.032253 5852 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 06:58:30.042491 5852 rpc_server.cc:307] RPC server started. Bound to: 127.19.52.67:35465
I20260812 06:58:30.042515 6001 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.52.67:35465 every 8 connection(s)
I20260812 06:58:30.043617 5852 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517687384214-19665-0/minicluster-data/ts-2/data/info.pb
I20260812 06:58:30.046347 19665 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu as pid 5852
Aug 12 06:58:30 dist-test-slave-65mx krb5kdc[4794](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517909, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.52.67@KRBTEST.COM for kudu/127.19.52.126@KRBTEST.COM
I20260812 06:58:30.056093 6005 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:58:30.047056 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58848 (local address 127.19.52.126:36401)
0812 06:58:30.047289 (+ 233us) server_negotiation.cc:206] Beginning negotiation
0812 06:58:30.047294 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 06:58:30.047486 (+ 192us) server_negotiation.cc:407] Connection header received
0812 06:58:30.047671 (+ 185us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:58:30.047673 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:58:30.047719 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:58:30.047812 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:58:30.048617 (+ 805us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:58:30.049402 (+ 785us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:58:30.050359 (+ 957us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:58:30.050596 (+ 237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:58:30.050713 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:58:30.052239 (+ 1526us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:58:30.052242 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:58:30.052244 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:58:30.052270 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:58:30.054114 (+ 1844us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:58:30.054601 (+ 487us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:58:30.054606 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:58:30.054608 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:58:30.054737 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:58:30.055026 (+ 289us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:58:30.055032 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:58:30.055037 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:58:30.055272 (+ 235us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:58:30.055457 (+ 185us) server_negotiation.cc:1035] Waiting for connection context
0812 06:58:30.055738 (+ 281us) server_negotiation.cc:299] Negotiation successful
0812 06:58:30.055890 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":158,"thread_start_us":79,"threads_started":1}
I20260812 06:58:30.059434 5687 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:58:30.045651 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.52.67:52665 (local address 127.19.52.126:36401)
0812 06:58:30.045827 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0812 06:58:30.045838 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0812 06:58:30.046791 (+ 953us) server_negotiation.cc:407] Connection header received
0812 06:58:30.048325 (+ 1534us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:58:30.048329 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:58:30.048377 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:58:30.048452 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:58:30.049927 (+ 1475us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:58:30.050428 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:58:30.051824 (+ 1396us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:58:30.052056 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:58:30.052175 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:58:30.055373 (+ 3198us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:58:30.055377 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:58:30.055379 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:58:30.055407 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:58:30.057191 (+ 1784us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:58:30.057827 (+ 636us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:58:30.057832 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:58:30.057834 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:58:30.057920 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:58:30.058328 (+ 408us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:58:30.058332 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:58:30.058334 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:58:30.058512 (+ 178us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:58:30.058623 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0812 06:58:30.059148 (+ 525us) server_negotiation.cc:299] Negotiation successful
0812 06:58:30.059267 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260812 06:58:30.060312 6004 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:58:30.046042 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.126:36401 (local address 127.19.52.67:52665)
0812 06:58:30.046604 (+ 562us) negotiation.cc:107] Waiting for socket to connect
0812 06:58:30.046666 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0812 06:58:30.048117 (+ 1451us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:58:30.048658 (+ 541us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:58:30.048665 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:58:30.049245 (+ 580us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:58:30.049764 (+ 519us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:58:30.049778 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:58:30.050597 (+ 819us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:58:30.050605 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:58:30.051689 (+ 1084us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:58:30.051699 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:58:30.052225 (+ 526us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:58:30.052229 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:58:30.052400 (+ 171us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:58:30.053138 (+ 738us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 06:58:30.053155 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:58:30.055162 (+ 2007us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 06:58:30.057308 (+ 2146us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:58:30.057314 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:58:30.057332 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:58:30.057718 (+ 386us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:58:30.058028 (+ 310us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:58:30.058031 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:58:30.058033 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:58:30.058226 (+ 193us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:58:30.058625 (+ 399us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 06:58:30.058635 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 06:58:30.058905 (+ 270us) client_negotiation.cc:769] Sending connection context
0812 06:58:30.059108 (+ 203us) client_negotiation.cc:240] Negotiation successful
0812 06:58:30.059471 (+ 363us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":307,"thread_start_us":117,"threads_started":1}
I20260812 06:58:30.061498 6002 heartbeater.cc:347] Connected to a master server at 127.19.52.126:36401
I20260812 06:58:30.061731 6002 heartbeater.cc:464] Registering TS with master...
I20260812 06:58:30.062191 6002 heartbeater.cc:511] Master 127.19.52.126:36401 requested a full tablet report, sending...
I20260812 06:58:30.063184 5465 ts_manager.cc:194] Registered new tserver with Master: 666220fa0cdf403dad047d3011fed16f (127.19.52.67:35465)
I20260812 06:58:30.063705 5465 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.52.67@KRBTEST.COM'} at 127.19.52.67:52665
I20260812 06:58:30.070240 19665 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 06:58:30.074822 5687 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:58:30.070902 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58862 (local address 127.19.52.126:36401)
0812 06:58:30.071120 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0812 06:58:30.071125 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 06:58:30.071286 (+ 161us) server_negotiation.cc:407] Connection header received
0812 06:58:30.071494 (+ 208us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:58:30.071497 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:58:30.071605 (+ 108us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:58:30.071722 (+ 117us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 06:58:30.071990 (+ 268us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:58:30.072621 (+ 631us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:58:30.073561 (+ 940us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:58:30.073785 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:58:30.073895 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:58:30.074249 (+ 354us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 06:58:30.074391 (+ 142us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_AUTHENTICATION_TOKEN: Not authorized: token expired
0812 06:58:30.074677 (+ 286us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:58862: token expired
Metrics: {"server-negotiator.queue_time_us":114}
W20260812 06:58:30.074919 5687 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:58862: token expired
W20260812 06:58:30.075249 5392 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.19.52.126:36401)
I20260812 06:58:30.083279 5687 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:58:30.075702 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58866 (local address 127.19.52.126:36401)
0812 06:58:30.075844 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0812 06:58:30.075847 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 06:58:30.075866 (+ 19us) server_negotiation.cc:407] Connection header received
0812 06:58:30.075897 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:58:30.075899 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:58:30.075936 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:58:30.076013 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:58:30.076711 (+ 698us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:58:30.077250 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:58:30.078071 (+ 821us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:58:30.078312 (+ 241us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:58:30.078404 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:58:30.079984 (+ 1580us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:58:30.079987 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:58:30.079989 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:58:30.080013 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:58:30.081608 (+ 1595us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:58:30.082054 (+ 446us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:58:30.082057 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:58:30.082058 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:58:30.082150 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:58:30.082406 (+ 256us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:58:30.082408 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:58:30.082409 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:58:30.082664 (+ 255us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:58:30.082811 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0812 06:58:30.083058 (+ 247us) server_negotiation.cc:299] Negotiation successful
0812 06:58:30.083161 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260812 06:58:30.086193 5465 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58866:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 06:58:30.089674 5465 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 06:58:30.098311 5608 tablet_service.cc:1467] Processing CreateTablet for tablet 63646b133f6c4739aa097080c3cf5ccf (DEFAULT_TABLE table=test-table [id=a3ba41716ac2425da93c654f50a98ce3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 06:58:30.099076 5608 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 63646b133f6c4739aa097080c3cf5ccf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 06:58:30.102300 5559 tablet_bootstrap.cc:492] T 63646b133f6c4739aa097080c3cf5ccf P aa4029c643b448ad9e197e92749c8fe6: Bootstrap starting.
I20260812 06:58:30.103293 5694 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:58:30.097105 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.67:35465 (local address 127.0.0.1:57890)
0812 06:58:30.097284 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0812 06:58:30.097299 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 06:58:30.097384 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:58:30.098489 (+ 1105us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:58:30.098492 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:58:30.098510 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 06:58:30.098632 (+ 122us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:58:30.098639 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:58:30.100362 (+ 1723us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:58:30.100365 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:58:30.101692 (+ 1327us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:58:30.101703 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:58:30.102795 (+ 1092us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:58:30.102798 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:58:30.102919 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:58:30.102978 (+ 59us) client_negotiation.cc:769] Sending connection context
0812 06:58:30.103039 (+ 61us) client_negotiation.cc:240] Negotiation successful
0812 06:58:30.103137 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":104,"mutex_wait_us":67}
I20260812 06:58:30.103335 5559 tablet_bootstrap.cc:654] T 63646b133f6c4739aa097080c3cf5ccf P aa4029c643b448ad9e197e92749c8fe6: Neither blocks nor log segments found. Creating new log.
I20260812 06:58:30.103470 6014 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:58:30.097521 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.66:34423 (local address 127.0.0.1:33738)
0812 06:58:30.097916 (+ 395us) negotiation.cc:107] Waiting for socket to connect
0812 06:58:30.097931 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 06:58:30.098045 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:58:30.098715 (+ 670us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:58:30.098719 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:58:30.098739 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 06:58:30.098826 (+ 87us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:58:30.098831 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:58:30.100362 (+ 1531us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:58:30.100365 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:58:30.101400 (+ 1035us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:58:30.101411 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:58:30.102778 (+ 1367us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:58:30.102781 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:58:30.103178 (+ 397us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:58:30.103200 (+ 22us) client_negotiation.cc:769] Sending connection context
0812 06:58:30.103268 (+ 68us) client_negotiation.cc:240] Negotiation successful
0812 06:58:30.103343 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":326,"mutex_wait_us":355,"thread_start_us":120,"threads_started":1}
I20260812 06:58:30.103957 6012 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:58:30.097304 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57890 (local address 127.19.52.67:35465)
0812 06:58:30.097955 (+ 651us) server_negotiation.cc:206] Beginning negotiation
0812 06:58:30.097979 (+ 24us) server_negotiation.cc:399] Waiting for connection header
0812 06:58:30.098006 (+ 27us) server_negotiation.cc:407] Connection header received
0812 06:58:30.098095 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:58:30.098100 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:58:30.098329 (+ 229us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:58:30.098526 (+ 197us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 06:58:30.098763 (+ 237us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:58:30.100220 (+ 1457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:58:30.101845 (+ 1625us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:58:30.102662 (+ 817us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:58:30.103472 (+ 810us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:58:30.103663 (+ 191us) server_negotiation.cc:1035] Waiting for connection context
0812 06:58:30.103710 (+ 47us) server_negotiation.cc:299] Negotiation successful
0812 06:58:30.103803 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":542,"thread_start_us":192,"threads_started":1}
I20260812 06:58:30.103956 6017 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:58:30.097713 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33738 (local address 127.19.52.66:34423)
0812 06:58:30.098175 (+ 462us) server_negotiation.cc:206] Beginning negotiation
0812 06:58:30.098186 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0812 06:58:30.098209 (+ 23us) server_negotiation.cc:407] Connection header received
0812 06:58:30.098315 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:58:30.098320 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:58:30.098524 (+ 204us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:58:30.098681 (+ 157us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 06:58:30.098963 (+ 282us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:58:30.100230 (+ 1267us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:58:30.101568 (+ 1338us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:58:30.102414 (+ 846us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:58:30.102532 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:58:30.102702 (+ 170us) server_negotiation.cc:1035] Waiting for connection context
0812 06:58:30.103703 (+ 1001us) server_negotiation.cc:299] Negotiation successful
0812 06:58:30.103807 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":368,"thread_start_us":123,"threads_started":1}
I20260812 06:58:30.104983 5559 tablet_bootstrap.cc:492] T 63646b133f6c4739aa097080c3cf5ccf P aa4029c643b448ad9e197e92749c8fe6: No bootstrap required, opened a new log
I20260812 06:58:30.105084 5559 ts_tablet_manager.cc:1397] T 63646b133f6c4739aa097080c3cf5ccf P aa4029c643b448ad9e197e92749c8fe6: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.001s
I20260812 06:58:30.105720 5559 raft_consensus.cc:348] T 63646b133f6c4739aa097080c3cf5ccf P aa4029c643b448ad9e197e92749c8fe6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "666220fa0cdf403dad047d3011fed16f" member_type: VOTER last_known_addr { host: "127.19.52.67" port: 35465 } } peers { permanent_uuid: "25ef252218c24d6cae8e3f5efff644ea" member_type: VOTER last_known_addr { host: "127.19.52.66" port: 34423 } } peers { permanent_uuid: "aa4029c643b448ad9e197e92749c8fe6" member_type: VOTER last_known_addr { host: "127.19.52.65" port: 41283 } }
I20260812 06:58:30.105856 5559 raft_consensus.cc:374] T 63646b133f6c4739aa097080c3cf5ccf P aa4029c643b448ad9e197e92749c8fe6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 06:58:30.105896 5559 raft_consensus.cc:729] T 63646b133f6c4739aa097080c3cf5ccf P aa4029c643b448ad9e197e92749c8fe6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aa4029c643b448ad9e197e92749c8fe6, State: Initialized, Role: FOLLOWER
I20260812 06:58:30.106024 5559 consensus_queue.cc:260] T 63646b133f6c4739aa097080c3cf5ccf P aa4029c643b448ad9e197e92749c8fe6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "666220fa0cdf403dad047d3011fed16f" member_type: VOTER last_known_addr { host: "127.19.52.67" port: 35465 } } peers { permanent_uuid: "25ef252218c24d6cae8e3f5efff644ea" member_type: VOTER last_known_addr { host: "127.19.52.66" port: 34423 } } peers { permanent_uuid: "aa4029c643b448ad9e197e92749c8fe6" member_type: VOTER last_known_addr { host: "127.19.52.65" port: 41283 } }
I20260812 06:58:30.106288 5559 ts_tablet_manager.cc:1428] T 63646b133f6c4739aa097080c3cf5ccf P aa4029c643b448ad9e197e92749c8fe6: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20260812 06:58:30.107215 5922 tablet_service.cc:1467] Processing CreateTablet for tablet 63646b133f6c4739aa097080c3cf5ccf (DEFAULT_TABLE table=test-table [id=a3ba41716ac2425da93c654f50a98ce3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 06:58:30.107220 5772 tablet_service.cc:1467] Processing CreateTablet for tablet 63646b133f6c4739aa097080c3cf5ccf (DEFAULT_TABLE table=test-table [id=a3ba41716ac2425da93c654f50a98ce3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 06:58:30.108202 5772 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 63646b133f6c4739aa097080c3cf5ccf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 06:58:30.108258 5922 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 63646b133f6c4739aa097080c3cf5ccf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 06:58:30.115742 6020 tablet_bootstrap.cc:492] T 63646b133f6c4739aa097080c3cf5ccf P 25ef252218c24d6cae8e3f5efff644ea: Bootstrap starting.
I20260812 06:58:30.115974 6021 tablet_bootstrap.cc:492] T 63646b133f6c4739aa097080c3cf5ccf P 666220fa0cdf403dad047d3011fed16f: Bootstrap starting.
I20260812 06:58:30.118696 6020 tablet_bootstrap.cc:654] T 63646b133f6c4739aa097080c3cf5ccf P 25ef252218c24d6cae8e3f5efff644ea: Neither blocks nor log segments found. Creating new log.
I20260812 06:58:30.118779 6021 tablet_bootstrap.cc:654] T 63646b133f6c4739aa097080c3cf5ccf P 666220fa0cdf403dad047d3011fed16f: Neither blocks nor log segments found. Creating new log.
I20260812 06:58:30.119477 6021 log.cc:824] T 63646b133f6c4739aa097080c3cf5ccf P 666220fa0cdf403dad047d3011fed16f: Log is configured to *not* fsync() on all Append() calls
I20260812 06:58:30.119477 6020 log.cc:824] T 63646b133f6c4739aa097080c3cf5ccf P 25ef252218c24d6cae8e3f5efff644ea: Log is configured to *not* fsync() on all Append() calls
I20260812 06:58:30.121587 6021 tablet_bootstrap.cc:492] T 63646b133f6c4739aa097080c3cf5ccf P 666220fa0cdf403dad047d3011fed16f: No bootstrap required, opened a new log
I20260812 06:58:30.121815 6021 ts_tablet_manager.cc:1397] T 63646b133f6c4739aa097080c3cf5ccf P 666220fa0cdf403dad047d3011fed16f: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260812 06:58:30.123131 6020 tablet_bootstrap.cc:492] T 63646b133f6c4739aa097080c3cf5ccf P 25ef252218c24d6cae8e3f5efff644ea: No bootstrap required, opened a new log
I20260812 06:58:30.123328 6020 ts_tablet_manager.cc:1397] T 63646b133f6c4739aa097080c3cf5ccf P 25ef252218c24d6cae8e3f5efff644ea: Time spent bootstrapping tablet: real 0.008s user 0.004s sys 0.000s
I20260812 06:58:30.125953 6021 raft_consensus.cc:348] T 63646b133f6c4739aa097080c3cf5ccf P 666220fa0cdf403dad047d3011fed16f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "666220fa0cdf403dad047d3011fed16f" member_type: VOTER last_known_addr { host: "127.19.52.67" port: 35465 } } peers { permanent_uuid: "25ef252218c24d6cae8e3f5efff644ea" member_type: VOTER last_known_addr { host: "127.19.52.66" port: 34423 } } peers { permanent_uuid: "aa4029c643b448ad9e197e92749c8fe6" member_type: VOTER last_known_addr { host: "127.19.52.65" port: 41283 } }
I20260812 06:58:30.126298 6021 raft_consensus.cc:374] T 63646b133f6c4739aa097080c3cf5ccf P 666220fa0cdf403dad047d3011fed16f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 06:58:30.126350 6021 raft_consensus.cc:729] T 63646b133f6c4739aa097080c3cf5ccf P 666220fa0cdf403dad047d3011fed16f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 666220fa0cdf403dad047d3011fed16f, State: Initialized, Role: FOLLOWER
I20260812 06:58:30.126842 6021 consensus_queue.cc:260] T 63646b133f6c4739aa097080c3cf5ccf P 666220fa0cdf403dad047d3011fed16f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "666220fa0cdf403dad047d3011fed16f" member_type: VOTER last_known_addr { host: "127.19.52.67" port: 35465 } } peers { permanent_uuid: "25ef252218c24d6cae8e3f5efff644ea" member_type: VOTER last_known_addr { host: "127.19.52.66" port: 34423 } } peers { permanent_uuid: "aa4029c643b448ad9e197e92749c8fe6" member_type: VOTER last_known_addr { host: "127.19.52.65" port: 41283 } }
I20260812 06:58:30.127197 6020 raft_consensus.cc:348] T 63646b133f6c4739aa097080c3cf5ccf P 25ef252218c24d6cae8e3f5efff644ea [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "666220fa0cdf403dad047d3011fed16f" member_type: VOTER last_known_addr { host: "127.19.52.67" port: 35465 } } peers { permanent_uuid: "25ef252218c24d6cae8e3f5efff644ea" member_type: VOTER last_known_addr { host: "127.19.52.66" port: 34423 } } peers { permanent_uuid: "aa4029c643b448ad9e197e92749c8fe6" member_type: VOTER last_known_addr { host: "127.19.52.65" port: 41283 } }
I20260812 06:58:30.127465 6020 raft_consensus.cc:374] T 63646b133f6c4739aa097080c3cf5ccf P 25ef252218c24d6cae8e3f5efff644ea [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 06:58:30.127511 6020 raft_consensus.cc:729] T 63646b133f6c4739aa097080c3cf5ccf P 25ef252218c24d6cae8e3f5efff644ea [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 25ef252218c24d6cae8e3f5efff644ea, State: Initialized, Role: FOLLOWER
I20260812 06:58:30.127625 6002 heartbeater.cc:503] Master 127.19.52.126:36401 was elected leader, sending a full tablet report...
I20260812 06:58:30.127931 6020 consensus_queue.cc:260] T 63646b133f6c4739aa097080c3cf5ccf P 25ef252218c24d6cae8e3f5efff644ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "666220fa0cdf403dad047d3011fed16f" member_type: VOTER last_known_addr { host: "127.19.52.67" port: 35465 } } peers { permanent_uuid: "25ef252218c24d6cae8e3f5efff644ea" member_type: VOTER last_known_addr { host: "127.19.52.66" port: 34423 } } peers { permanent_uuid: "aa4029c643b448ad9e197e92749c8fe6" member_type: VOTER last_known_addr { host: "127.19.52.65" port: 41283 } }
I20260812 06:58:30.128067 6021 ts_tablet_manager.cc:1428] T 63646b133f6c4739aa097080c3cf5ccf P 666220fa0cdf403dad047d3011fed16f: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260812 06:58:30.128964 5848 heartbeater.cc:503] Master 127.19.52.126:36401 was elected leader, sending a full tablet report...
I20260812 06:58:30.128968 6020 ts_tablet_manager.cc:1428] T 63646b133f6c4739aa097080c3cf5ccf P 25ef252218c24d6cae8e3f5efff644ea: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
W20260812 06:58:30.134541 5686 tablet.cc:2369] T 63646b133f6c4739aa097080c3cf5ccf P aa4029c643b448ad9e197e92749c8fe6: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 06:58:30.294719 6003 tablet.cc:2369] T 63646b133f6c4739aa097080c3cf5ccf P 666220fa0cdf403dad047d3011fed16f: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 06:58:30.309688 5692 raft_consensus.cc:482] T 63646b133f6c4739aa097080c3cf5ccf P aa4029c643b448ad9e197e92749c8fe6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 06:58:30.310062 5692 raft_consensus.cc:504] T 63646b133f6c4739aa097080c3cf5ccf P aa4029c643b448ad9e197e92749c8fe6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "666220fa0cdf403dad047d3011fed16f" member_type: VOTER last_known_addr { host: "127.19.52.67" port: 35465 } } peers { permanent_uuid: "25ef252218c24d6cae8e3f5efff644ea" member_type: VOTER last_known_addr { host: "127.19.52.66" port: 34423 } } peers { permanent_uuid: "aa4029c643b448ad9e197e92749c8fe6" member_type: VOTER last_known_addr { host: "127.19.52.65" port: 41283 } }
I20260812 06:58:30.311411 5692 leader_election.cc:288] T 63646b133f6c4739aa097080c3cf5ccf P aa4029c643b448ad9e197e92749c8fe6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 666220fa0cdf403dad047d3011fed16f (127.19.52.67:35465), 25ef252218c24d6cae8e3f5efff644ea (127.19.52.66:34423)
I20260812 06:58:30.315892 6017 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:58:30.311900 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.52.65:52851 (local address 127.19.52.66:34423)
0812 06:58:30.312040 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0812 06:58:30.312044 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 06:58:30.312286 (+ 242us) server_negotiation.cc:407] Connection header received
0812 06:58:30.312515 (+ 229us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:58:30.312518 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:58:30.312616 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:58:30.312707 (+ 91us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 06:58:30.313056 (+ 349us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:58:30.313571 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:58:30.314844 (+ 1273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:58:30.315227 (+ 383us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:58:30.315325 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:58:30.315398 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0812 06:58:30.315679 (+ 281us) server_negotiation.cc:299] Negotiation successful
0812 06:58:30.315756 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260812 06:58:30.315927 6026 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:58:30.311737 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.67:35465 (local address 127.19.52.65:33677)
0812 06:58:30.312205 (+ 468us) negotiation.cc:107] Waiting for socket to connect
0812 06:58:30.312217 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 06:58:30.312361 (+ 144us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:58:30.312737 (+ 376us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:58:30.312740 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:58:30.312757 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 06:58:30.312941 (+ 184us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:58:30.312948 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:58:30.313811 (+ 863us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:58:30.313813 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:58:30.314668 (+ 855us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:58:30.314677 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:58:30.315449 (+ 772us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:58:30.315453 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:58:30.315537 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:58:30.315572 (+ 35us) client_negotiation.cc:769] Sending connection context
0812 06:58:30.315669 (+ 97us) client_negotiation.cc:240] Negotiation successful
0812 06:58:30.315749 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":408,"mutex_wait_us":132,"thread_start_us":97,"threads_started":1}
I20260812 06:58:30.315920 6012 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:58:30.311849 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.52.65:33677 (local address 127.19.52.67:35465)
0812 06:58:30.311998 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0812 06:58:30.312004 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 06:58:30.312356 (+ 352us) server_negotiation.cc:407] Connection header received
0812 06:58:30.312494 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:58:30.312498 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:58:30.312605 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:58:30.312712 (+ 107us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 06:58:30.313097 (+ 385us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:58:30.313683 (+ 586us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:58:30.314808 (+ 1125us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:58:30.315305 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:58:30.315440 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:58:30.315495 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0812 06:58:30.315707 (+ 212us) server_negotiation.cc:299] Negotiation successful
0812 06:58:30.315793 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260812 06:58:30.316174 6027 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:58:30.311737 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.66:34423 (local address 127.19.52.65:52851)
0812 06:58:30.312163 (+ 426us) negotiation.cc:107] Waiting for socket to connect
0812 06:58:30.312181 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0812 06:58:30.312343 (+ 162us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:58:30.312722 (+ 379us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:58:30.312725 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:58:30.312744 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 06:58:30.312931 (+ 187us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:58:30.312937 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:58:30.313688 (+ 751us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:58:30.313692 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:58:30.314680 (+ 988us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:58:30.314690 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:58:30.315361 (+ 671us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:58:30.315368 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:58:30.315494 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:58:30.315559 (+ 65us) client_negotiation.cc:769] Sending connection context
0812 06:58:30.315628 (+ 69us) client_negotiation.cc:240] Negotiation successful
0812 06:58:30.315743 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":342,"mutex_wait_us":27,"spinlock_wait_cycles":14208,"thread_start_us":113,"threads_started":1}
I20260812 06:58:30.316799 5946 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "63646b133f6c4739aa097080c3cf5ccf" candidate_uuid: "aa4029c643b448ad9e197e92749c8fe6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "666220fa0cdf403dad047d3011fed16f" is_pre_election: true
I20260812 06:58:30.316871 5794 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "63646b133f6c4739aa097080c3cf5ccf" candidate_uuid: "aa4029c643b448ad9e197e92749c8fe6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "25ef252218c24d6cae8e3f5efff644ea" is_pre_election: true
I20260812 06:58:30.317189 5946 raft_consensus.cc:2454] T 63646b133f6c4739aa097080c3cf5ccf P 666220fa0cdf403dad047d3011fed16f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aa4029c643b448ad9e197e92749c8fe6 in term 0.
I20260812 06:58:30.317230 5794 raft_consensus.cc:2454] T 63646b133f6c4739aa097080c3cf5ccf P 25ef252218c24d6cae8e3f5efff644ea [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aa4029c643b448ad9e197e92749c8fe6 in term 0.
I20260812 06:58:30.317726 5554 leader_election.cc:302] T 63646b133f6c4739aa097080c3cf5ccf P aa4029c643b448ad9e197e92749c8fe6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 666220fa0cdf403dad047d3011fed16f, aa4029c643b448ad9e197e92749c8fe6; no voters:
I20260812 06:58:30.318064 5692 raft_consensus.cc:2781] T 63646b133f6c4739aa097080c3cf5ccf P aa4029c643b448ad9e197e92749c8fe6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 06:58:30.318159 5692 raft_consensus.cc:482] T 63646b133f6c4739aa097080c3cf5ccf P aa4029c643b448ad9e197e92749c8fe6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 06:58:30.318217 5692 raft_consensus.cc:3037] T 63646b133f6c4739aa097080c3cf5ccf P aa4029c643b448ad9e197e92749c8fe6 [term 0 FOLLOWER]: Advancing to term 1
I20260812 06:58:30.319346 5692 raft_consensus.cc:504] T 63646b133f6c4739aa097080c3cf5ccf P aa4029c643b448ad9e197e92749c8fe6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "666220fa0cdf403dad047d3011fed16f" member_type: VOTER last_known_addr { host: "127.19.52.67" port: 35465 } } peers { permanent_uuid: "25ef252218c24d6cae8e3f5efff644ea" member_type: VOTER last_known_addr { host: "127.19.52.66" port: 34423 } } peers { permanent_uuid: "aa4029c643b448ad9e197e92749c8fe6" member_type: VOTER last_known_addr { host: "127.19.52.65" port: 41283 } }
I20260812 06:58:30.319773 5692 leader_election.cc:288] T 63646b133f6c4739aa097080c3cf5ccf P aa4029c643b448ad9e197e92749c8fe6 [CANDIDATE]: Term 1 election: Requested vote from peers 666220fa0cdf403dad047d3011fed16f (127.19.52.67:35465), 25ef252218c24d6cae8e3f5efff644ea (127.19.52.66:34423)
I20260812 06:58:30.320192 5946 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "63646b133f6c4739aa097080c3cf5ccf" candidate_uuid: "aa4029c643b448ad9e197e92749c8fe6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "666220fa0cdf403dad047d3011fed16f"
I20260812 06:58:30.320217 5794 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "63646b133f6c4739aa097080c3cf5ccf" candidate_uuid: "aa4029c643b448ad9e197e92749c8fe6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "25ef252218c24d6cae8e3f5efff644ea"
I20260812 06:58:30.320319 5946 raft_consensus.cc:3037] T 63646b133f6c4739aa097080c3cf5ccf P 666220fa0cdf403dad047d3011fed16f [term 0 FOLLOWER]: Advancing to term 1
I20260812 06:58:30.320354 5794 raft_consensus.cc:3037] T 63646b133f6c4739aa097080c3cf5ccf P 25ef252218c24d6cae8e3f5efff644ea [term 0 FOLLOWER]: Advancing to term 1
I20260812 06:58:30.321535 5794 raft_consensus.cc:2454] T 63646b133f6c4739aa097080c3cf5ccf P 25ef252218c24d6cae8e3f5efff644ea [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aa4029c643b448ad9e197e92749c8fe6 in term 1.
I20260812 06:58:30.321563 5946 raft_consensus.cc:2454] T 63646b133f6c4739aa097080c3cf5ccf P 666220fa0cdf403dad047d3011fed16f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aa4029c643b448ad9e197e92749c8fe6 in term 1.
I20260812 06:58:30.321882 5554 leader_election.cc:302] T 63646b133f6c4739aa097080c3cf5ccf P aa4029c643b448ad9e197e92749c8fe6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 666220fa0cdf403dad047d3011fed16f, aa4029c643b448ad9e197e92749c8fe6; no voters:
I20260812 06:58:30.322080 5692 raft_consensus.cc:2781] T 63646b133f6c4739aa097080c3cf5ccf P aa4029c643b448ad9e197e92749c8fe6 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 06:58:30.322268 5692 raft_consensus.cc:686] T 63646b133f6c4739aa097080c3cf5ccf P aa4029c643b448ad9e197e92749c8fe6 [term 1 LEADER]: Becoming Leader. State: Replica: aa4029c643b448ad9e197e92749c8fe6, State: Running, Role: LEADER
I20260812 06:58:30.322552 5692 consensus_queue.cc:237] T 63646b133f6c4739aa097080c3cf5ccf P aa4029c643b448ad9e197e92749c8fe6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "666220fa0cdf403dad047d3011fed16f" member_type: VOTER last_known_addr { host: "127.19.52.67" port: 35465 } } peers { permanent_uuid: "25ef252218c24d6cae8e3f5efff644ea" member_type: VOTER last_known_addr { host: "127.19.52.66" port: 34423 } } peers { permanent_uuid: "aa4029c643b448ad9e197e92749c8fe6" member_type: VOTER last_known_addr { host: "127.19.52.65" port: 41283 } }
I20260812 06:58:30.325562 5466 catalog_manager.cc:5495] T 63646b133f6c4739aa097080c3cf5ccf P aa4029c643b448ad9e197e92749c8fe6 reported cstate change: term changed from 0 to 1, leader changed from <none> to aa4029c643b448ad9e197e92749c8fe6 (127.19.52.65). New cstate: current_term: 1 leader_uuid: "aa4029c643b448ad9e197e92749c8fe6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "666220fa0cdf403dad047d3011fed16f" member_type: VOTER last_known_addr { host: "127.19.52.67" port: 35465 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "25ef252218c24d6cae8e3f5efff644ea" member_type: VOTER last_known_addr { host: "127.19.52.66" port: 34423 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "aa4029c643b448ad9e197e92749c8fe6" member_type: VOTER last_known_addr { host: "127.19.52.65" port: 41283 } health_report { overall_health: HEALTHY } } }
W20260812 06:58:30.340772 5849 tablet.cc:2369] T 63646b133f6c4739aa097080c3cf5ccf P 25ef252218c24d6cae8e3f5efff644ea: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 06:58:30.343309 6031 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:58:30.339396 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44596 (local address 127.19.52.65:41283)
0812 06:58:30.339691 (+ 295us) server_negotiation.cc:206] Beginning negotiation
0812 06:58:30.339703 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0812 06:58:30.339717 (+ 14us) server_negotiation.cc:407] Connection header received
0812 06:58:30.339791 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:58:30.339793 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:58:30.339848 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:58:30.339961 (+ 113us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 06:58:30.340203 (+ 242us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:58:30.340780 (+ 577us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:58:30.341864 (+ 1084us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:58:30.342116 (+ 252us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:58:30.342200 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:58:30.342479 (+ 279us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 06:58:30.342897 (+ 418us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 06:58:30.342984 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0812 06:58:30.343125 (+ 141us) server_negotiation.cc:299] Negotiation successful
0812 06:58:30.343182 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":222,"thread_start_us":138,"threads_started":1}
I20260812 06:58:30.347645 5794 raft_consensus.cc:1261] T 63646b133f6c4739aa097080c3cf5ccf P 25ef252218c24d6cae8e3f5efff644ea [term 1 FOLLOWER]: Refusing update from remote peer aa4029c643b448ad9e197e92749c8fe6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 06:58:30.347646 5946 raft_consensus.cc:1261] T 63646b133f6c4739aa097080c3cf5ccf P 666220fa0cdf403dad047d3011fed16f [term 1 FOLLOWER]: Refusing update from remote peer aa4029c643b448ad9e197e92749c8fe6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 06:58:30.348564 6028 consensus_queue.cc:1035] T 63646b133f6c4739aa097080c3cf5ccf P aa4029c643b448ad9e197e92749c8fe6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "25ef252218c24d6cae8e3f5efff644ea" member_type: VOTER last_known_addr { host: "127.19.52.66" port: 34423 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 06:58:30.348784 5692 consensus_queue.cc:1035] T 63646b133f6c4739aa097080c3cf5ccf P aa4029c643b448ad9e197e92749c8fe6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "666220fa0cdf403dad047d3011fed16f" member_type: VOTER last_known_addr { host: "127.19.52.67" port: 35465 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 06:58:30.357293 6032 mvcc.cc:205] Tried to move back new op lower bound from 7317577360777195520 to 7317577360683745280. Current Snapshot: MvccSnapshot[applied={T|T < 7317577360777195520}]
I20260812 06:58:30.358259 6034 mvcc.cc:205] Tried to move back new op lower bound from 7317577360777195520 to 7317577360683745280. Current Snapshot: MvccSnapshot[applied={T|T < 7317577360777195520}]
I20260812 06:58:30.359807 6033 mvcc.cc:205] Tried to move back new op lower bound from 7317577360777195520 to 7317577360683745280. Current Snapshot: MvccSnapshot[applied={T|T < 7317577360777195520}]
I20260812 06:58:30.365896 5466 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58866:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 06:58:30.366102 5466 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58866:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 06:58:30.368995 5466 catalog_manager.cc:5782] T 00000000000000000000000000000000 P f4608298ae804b898d7b5b8077543057: Sending DeleteTablet for 3 replicas of tablet 63646b133f6c4739aa097080c3cf5ccf
I20260812 06:58:30.369848 5772 tablet_service.cc:1514] Processing DeleteTablet for tablet 63646b133f6c4739aa097080c3cf5ccf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 06:58:30 UTC) from {username='kudu', principal='kudu/127.19.52.126@KRBTEST.COM'} at 127.0.0.1:33738
I20260812 06:58:30.370065 5608 tablet_service.cc:1514] Processing DeleteTablet for tablet 63646b133f6c4739aa097080c3cf5ccf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 06:58:30 UTC) from {username='kudu', principal='kudu/127.19.52.126@KRBTEST.COM'} at 127.0.0.1:44584
I20260812 06:58:30.370090 19665 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu with pid 5508
I20260812 06:58:30.370770 5922 tablet_service.cc:1514] Processing DeleteTablet for tablet 63646b133f6c4739aa097080c3cf5ccf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 06:58:30 UTC) from {username='kudu', principal='kudu/127.19.52.126@KRBTEST.COM'} at 127.0.0.1:57890
I20260812 06:58:30.371939 6046 tablet_replica.cc:330] T 63646b133f6c4739aa097080c3cf5ccf P 666220fa0cdf403dad047d3011fed16f: stopping tablet replica
I20260812 06:58:30.372360 6045 tablet_replica.cc:330] T 63646b133f6c4739aa097080c3cf5ccf P 25ef252218c24d6cae8e3f5efff644ea: stopping tablet replica
I20260812 06:58:30.372347 6046 raft_consensus.cc:2229] T 63646b133f6c4739aa097080c3cf5ccf P 666220fa0cdf403dad047d3011fed16f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 06:58:30.372836 6046 raft_consensus.cc:2258] T 63646b133f6c4739aa097080c3cf5ccf P 666220fa0cdf403dad047d3011fed16f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 06:58:30.373535 6045 raft_consensus.cc:2229] T 63646b133f6c4739aa097080c3cf5ccf P 25ef252218c24d6cae8e3f5efff644ea [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 06:58:30.373909 6045 raft_consensus.cc:2258] T 63646b133f6c4739aa097080c3cf5ccf P 25ef252218c24d6cae8e3f5efff644ea [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 06:58:30.375586 6046 ts_tablet_manager.cc:1905] T 63646b133f6c4739aa097080c3cf5ccf P 666220fa0cdf403dad047d3011fed16f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 06:58:30.375965 6045 ts_tablet_manager.cc:1905] T 63646b133f6c4739aa097080c3cf5ccf P 25ef252218c24d6cae8e3f5efff644ea: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 06:58:30.378204 6045 ts_tablet_manager.cc:1918] T 63646b133f6c4739aa097080c3cf5ccf P 25ef252218c24d6cae8e3f5efff644ea: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 06:58:30.378332 6045 log.cc:1196] T 63646b133f6c4739aa097080c3cf5ccf P 25ef252218c24d6cae8e3f5efff644ea: Deleting WAL directory at /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517687384214-19665-0/minicluster-data/ts-1/wal/wals/63646b133f6c4739aa097080c3cf5ccf
I20260812 06:58:30.378738 6045 ts_tablet_manager.cc:1939] T 63646b133f6c4739aa097080c3cf5ccf P 25ef252218c24d6cae8e3f5efff644ea: Deleting consensus metadata
I20260812 06:58:30.379645 6046 ts_tablet_manager.cc:1918] T 63646b133f6c4739aa097080c3cf5ccf P 666220fa0cdf403dad047d3011fed16f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 06:58:30.379765 6046 log.cc:1196] T 63646b133f6c4739aa097080c3cf5ccf P 666220fa0cdf403dad047d3011fed16f: Deleting WAL directory at /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517687384214-19665-0/minicluster-data/ts-2/wal/wals/63646b133f6c4739aa097080c3cf5ccf
I20260812 06:58:30.380125 5454 catalog_manager.cc:4841] TS 25ef252218c24d6cae8e3f5efff644ea (127.19.52.66:34423): tablet 63646b133f6c4739aa097080c3cf5ccf (table test-table [id=a3ba41716ac2425da93c654f50a98ce3]) successfully deleted
I20260812 06:58:30.380188 6046 ts_tablet_manager.cc:1939] T 63646b133f6c4739aa097080c3cf5ccf P 666220fa0cdf403dad047d3011fed16f: Deleting consensus metadata
I20260812 06:58:30.381633 5453 catalog_manager.cc:4841] TS 666220fa0cdf403dad047d3011fed16f (127.19.52.67:35465): tablet 63646b133f6c4739aa097080c3cf5ccf (table test-table [id=a3ba41716ac2425da93c654f50a98ce3]) successfully deleted
I20260812 06:58:30.383378 19665 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu with pid 5689
W20260812 06:58:30.383337 5455 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.19.52.65:41283 (error 108)
I20260812 06:58:30.386543 6014 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:58:30.386038 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.65:41283 (local address 127.0.0.1:44608)
0812 06:58:30.386295 (+ 257us) negotiation.cc:107] Waiting for socket to connect
0812 06:58:30.386400 (+ 105us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.52.65:41283: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":142}
W20260812 06:58:30.386796 5455 catalog_manager.cc:4584] TS aa4029c643b448ad9e197e92749c8fe6 (127.19.52.65:41283): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 63646b133f6c4739aa097080c3cf5ccf: Network error: Client connection negotiation failed: client connection to 127.19.52.65:41283: connect: Connection refused (error 111)
I20260812 06:58:30.395321 19665 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu with pid 5852
I20260812 06:58:30.404134 19665 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu with pid 5430
I20260812 06:58:50.414997 19665 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu
/tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517687384214-19665-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517687384214-19665-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517687384214-19665-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517687384214-19665-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.52.126:36401
--webserver_interface=127.19.52.126
--webserver_port=40333
--keytab_file=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517687384214-19665-0/krb5kdc/kudu_127.19.52.126.keytab
--principal=kudu/127.19.52.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.52.84:35215
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.19.52.126:36401
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517687384214-19665-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517687384214-19665-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517687384214-19665-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 06:58:50.540444 6048 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:58:50.540773 6048 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:58:50.540817 6048 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 06:58:50.545015 6048 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260812 06:58:50.545073 6048 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260812 06:58:50.545089 6048 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260812 06:58:50.545104 6048 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 06:58:50.545122 6048 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 06:58:50.545184 6048 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 06:58:50.545219 6048 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 06:58:50.545257 6048 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 06:58:50.550729 6048 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.52.84:35215
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517687384214-19665-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517687384214-19665-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.19.52.126:36401
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517687384214-19665-0/krb5kdc/kudu_127.19.52.126.keytab
--principal=kudu/127.19.52.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.52.126:36401
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517687384214-19665-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.19.52.126
--webserver_port=40333
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6048
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517687384214-19665-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 06:15:54 UTC on ccycloud.quasar-rnfzce.root.comops.site
build id 4581989
I20260812 06:58:50.551952 6048 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:58:50.553083 6048 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 06:58:50.562562 6056 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:58:50.562585 6048 server_base.cc:1034] running on GCE node
W20260812 06:58:50.562541 6054 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:58:50.562813 6053 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:58:50.563516 6048 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:58:50.564908 6048 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:58:50.566138 6048 hybrid_clock.cc:648] HybridClock initialized: now 1786517930566122 us; error 41 us; skew 500 ppm
Aug 12 06:58:50 dist-test-slave-65mx krb5kdc[4794](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517930, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.52.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 06:58:50.569269 6048 init.cc:377] Logged in from keytab as kudu/127.19.52.126@KRBTEST.COM (short username kudu)
I20260812 06:58:50.570165 6048 webserver.cc:460] Webserver started at http://127.19.52.126:40333/ using document root <none> and password file <none>
I20260812 06:58:50.570698 6048 fs_manager.cc:359] Metadata directory not provided
I20260812 06:58:50.570760 6048 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:58:50.574867 6048 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260812 06:58:50.577198 6063 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 06:58:50.578477 6048 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 06:58:50.578588 6048 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517687384214-19665-0/minicluster-data/master-0/data,/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517687384214-19665-0/minicluster-data/master-0/wal
uuid: "f4608298ae804b898d7b5b8077543057"
format_stamp: "Formatted at 2026-08-12 06:58:08 on dist-test-slave-65mx"
server_key: "e1431a0315a64cc0d3a002aa1a879aa3"
server_key_iv: "05eea69e240961fe06492fbfcc00dbef"
server_key_version: "encryptionkey@0"
I20260812 06:58:50.579134 6048 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517687384214-19665-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517687384214-19665-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517687384214-19665-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:58:50.609876 6048 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 06:58:50.613272 6048 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 06:58:50.613440 6048 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:58:50.621202 6048 rpc_server.cc:307] RPC server started. Bound to: 127.19.52.126:36401
I20260812 06:58:50.621536 6127 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.52.126:36401 every 8 connection(s)
I20260812 06:58:50.622345 6048 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517687384214-19665-0/minicluster-data/master-0/data/info.pb
I20260812 06:58:50.624902 19665 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu as pid 6048
I20260812 06:58:50.625380 19665 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu
/tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517687384214-19665-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517687384214-19665-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517687384214-19665-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517687384214-19665-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.52.65:41283
--local_ip_for_outbound_sockets=127.19.52.65
--tserver_master_addrs=127.19.52.126:36401
--webserver_port=44717
--webserver_interface=127.19.52.65
--keytab_file=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517687384214-19665-0/krb5kdc/kudu_127.19.52.65.keytab
--principal=kudu/127.19.52.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.52.84:35215
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517687384214-19665-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517687384214-19665-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517687384214-19665-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 06:58:50.627084 6128 sys_catalog.cc:263] Verifying existing consensus state
I20260812 06:58:50.628902 6128 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f4608298ae804b898d7b5b8077543057: Bootstrap starting.
I20260812 06:58:50.647688 6128 log.cc:824] T 00000000000000000000000000000000 P f4608298ae804b898d7b5b8077543057: Log is configured to *not* fsync() on all Append() calls
I20260812 06:58:50.659230 6128 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f4608298ae804b898d7b5b8077543057: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=8 ignored=0} mutations{seen=8 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260812 06:58:50.659708 6128 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f4608298ae804b898d7b5b8077543057: Bootstrap complete.
I20260812 06:58:50.664126 6128 raft_consensus.cc:348] T 00000000000000000000000000000000 P f4608298ae804b898d7b5b8077543057 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f4608298ae804b898d7b5b8077543057" member_type: VOTER last_known_addr { host: "127.19.52.126" port: 36401 } }
I20260812 06:58:50.664597 6128 raft_consensus.cc:729] T 00000000000000000000000000000000 P f4608298ae804b898d7b5b8077543057 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: f4608298ae804b898d7b5b8077543057, State: Initialized, Role: FOLLOWER
I20260812 06:58:50.665174 6128 consensus_queue.cc:260] T 00000000000000000000000000000000 P f4608298ae804b898d7b5b8077543057 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f4608298ae804b898d7b5b8077543057" member_type: VOTER last_known_addr { host: "127.19.52.126" port: 36401 } }
I20260812 06:58:50.665302 6128 raft_consensus.cc:388] T 00000000000000000000000000000000 P f4608298ae804b898d7b5b8077543057 [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 06:58:50.665400 6128 raft_consensus.cc:482] T 00000000000000000000000000000000 P f4608298ae804b898d7b5b8077543057 [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 06:58:50.665525 6128 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f4608298ae804b898d7b5b8077543057 [term 2 FOLLOWER]: Advancing to term 3
I20260812 06:58:50.667573 6128 raft_consensus.cc:504] T 00000000000000000000000000000000 P f4608298ae804b898d7b5b8077543057 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f4608298ae804b898d7b5b8077543057" member_type: VOTER last_known_addr { host: "127.19.52.126" port: 36401 } }
I20260812 06:58:50.667922 6128 leader_election.cc:302] T 00000000000000000000000000000000 P f4608298ae804b898d7b5b8077543057 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f4608298ae804b898d7b5b8077543057; no voters:
I20260812 06:58:50.668344 6128 leader_election.cc:288] T 00000000000000000000000000000000 P f4608298ae804b898d7b5b8077543057 [CANDIDATE]: Term 3 election: Requested vote from peers
I20260812 06:58:50.668442 6133 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f4608298ae804b898d7b5b8077543057 [term 3 FOLLOWER]: Leader election won for term 3
I20260812 06:58:50.668681 6133 raft_consensus.cc:686] T 00000000000000000000000000000000 P f4608298ae804b898d7b5b8077543057 [term 3 LEADER]: Becoming Leader. State: Replica: f4608298ae804b898d7b5b8077543057, State: Running, Role: LEADER
I20260812 06:58:50.669044 6133 consensus_queue.cc:237] T 00000000000000000000000000000000 P f4608298ae804b898d7b5b8077543057 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f4608298ae804b898d7b5b8077543057" member_type: VOTER last_known_addr { host: "127.19.52.126" port: 36401 } }
I20260812 06:58:50.669723 6128 sys_catalog.cc:564] T 00000000000000000000000000000000 P f4608298ae804b898d7b5b8077543057 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 06:58:50.670054 6133 sys_catalog.cc:455] T 00000000000000000000000000000000 P f4608298ae804b898d7b5b8077543057 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f4608298ae804b898d7b5b8077543057. Latest consensus state: current_term: 3 leader_uuid: "f4608298ae804b898d7b5b8077543057" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f4608298ae804b898d7b5b8077543057" member_type: VOTER last_known_addr { host: "127.19.52.126" port: 36401 } } }
I20260812 06:58:50.670176 6133 sys_catalog.cc:458] T 00000000000000000000000000000000 P f4608298ae804b898d7b5b8077543057 [sys.catalog]: This master's current role is: LEADER
I20260812 06:58:50.670794 6134 sys_catalog.cc:455] T 00000000000000000000000000000000 P f4608298ae804b898d7b5b8077543057 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "f4608298ae804b898d7b5b8077543057" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f4608298ae804b898d7b5b8077543057" member_type: VOTER last_known_addr { host: "127.19.52.126" port: 36401 } } }
I20260812 06:58:50.670997 6134 sys_catalog.cc:458] T 00000000000000000000000000000000 P f4608298ae804b898d7b5b8077543057 [sys.catalog]: This master's current role is: LEADER
I20260812 06:58:50.674558 6150 catalog_manager.cc:1259] Loaded cluster ID: 6dd4eb238b9f4adabe01ec004ec213af
I20260812 06:58:50.674623 6150 catalog_manager.cc:1552] T 00000000000000000000000000000000 P f4608298ae804b898d7b5b8077543057: loading cluster ID for follower catalog manager: success
I20260812 06:58:50.676453 6150 catalog_manager.cc:1574] T 00000000000000000000000000000000 P f4608298ae804b898d7b5b8077543057: acquiring CA information for follower catalog manager: success
F20260812 06:58:50.677481 6150 token_signing_key.cc:66] Check failed: _s.ok() Bad status: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7f587d9a29ad google::LogMessage::Fail() at ??:0
@ 0x7f587d9a588a google::LogMessage::SendToLog() at ??:0
@ 0x7f587d9a24de google::LogMessage::Flush() at ??:0
@ 0x7f587d9a3a19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7f588066618d kudu::security::TokenSigningPrivateKey::TokenSigningPrivateKey() at ??:0
@ 0x7f5888307989 kudu::master::CatalogManager::LoadTspkEntries() at ??:0
@ 0x7f58882e4023 kudu::master::CatalogManager::PrepareFollowerTokenVerifier() at ??:0
@ 0x7f58882e454b kudu::master::CatalogManager::PrepareFollower() at ??:0
@ 0x7f58882dd652 kudu::master::CatalogManagerBgTasks::Run() at ??:0
@ 0x7f58882dc9e9 _ZZN4kudu6master21CatalogManagerBgTasks4InitEvENKUlvE_clEv at ??:0
@ 0x7f5888323541 _ZNSt17_Function_handlerIFvvEZN4kudu6master21CatalogManagerBgTasks4InitEvEUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x61d838 std::function<>::operator()() at ??:0
@ 0x7f587fbe5512 kudu::Thread::SuperviseThread() at ??:0
@ 0x7f58812296db start_thread at ??:0
@ 0x7f587c4a061f clone at ??:0
W20260812 06:58:50.764451 6130 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:58:50.764851 6130 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:58:50.764935 6130 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 06:58:50.769232 6130 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 06:58:50.769315 6130 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 06:58:50.769404 6130 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.52.65
I20260812 06:58:50.774912 6130 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.52.84:35215
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517687384214-19665-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517687384214-19665-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517687384214-19665-0/krb5kdc/kudu_127.19.52.65.keytab
--principal=kudu/127.19.52.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.52.65:41283
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517687384214-19665-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.19.52.65
--webserver_port=44717
--tserver_master_addrs=127.19.52.126:36401
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6130
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.52.65
--log_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517687384214-19665-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 06:15:54 UTC on ccycloud.quasar-rnfzce.root.comops.site
build id 4581989
I20260812 06:58:50.776080 6130 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:58:50.777217 6130 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 06:58:50.788136 6173 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:58:50.788190 6130 server_base.cc:1034] running on GCE node
W20260812 06:58:50.788369 6174 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:58:50.788112 6176 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:58:50.788832 6130 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:58:50.789537 6130 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:58:50.790730 6130 hybrid_clock.cc:648] HybridClock initialized: now 1786517930790715 us; error 38 us; skew 500 ppm
Aug 12 06:58:50 dist-test-slave-65mx krb5kdc[4794](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517930, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.52.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 06:58:50.793679 6130 init.cc:377] Logged in from keytab as kudu/127.19.52.65@KRBTEST.COM (short username kudu)
I20260812 06:58:50.794631 6130 webserver.cc:460] Webserver started at http://127.19.52.65:44717/ using document root <none> and password file <none>
I20260812 06:58:50.795207 6130 fs_manager.cc:359] Metadata directory not provided
I20260812 06:58:50.795271 6130 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:58:50.799387 6130 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.000s
I20260812 06:58:50.801738 6183 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 06:58:50.803140 6130 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 06:58:50.803241 6130 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517687384214-19665-0/minicluster-data/ts-0/data,/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517687384214-19665-0/minicluster-data/ts-0/wal
uuid: "aa4029c643b448ad9e197e92749c8fe6"
format_stamp: "Formatted at 2026-08-12 06:58:08 on dist-test-slave-65mx"
server_key: "8850f5c2ac7fdd0cd582238231808ff3"
server_key_iv: "2ac23c925edf745076c39d522dc5cf21"
server_key_version: "encryptionkey@0"
I20260812 06:58:50.803830 6130 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517687384214-19665-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517687384214-19665-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517687384214-19665-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:58:50.821743 6130 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 06:58:50.825307 6130 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 06:58:50.825524 6130 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:58:50.826218 6130 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 06:58:50.827368 6190 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260812 06:58:50.831545 6130 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260812 06:58:50.831621 6130 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.004s user 0.001s sys 0.000s
I20260812 06:58:50.831671 6130 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260812 06:58:50.833485 6130 ts_tablet_manager.cc:610] Registered 1 tablets
I20260812 06:58:50.833546 6130 ts_tablet_manager.cc:589] Time spent register tablets: real 0.002s user 0.001s sys 0.000s
I20260812 06:58:50.833798 6190 tablet_bootstrap.cc:492] T 63646b133f6c4739aa097080c3cf5ccf P aa4029c643b448ad9e197e92749c8fe6: Bootstrap starting.
I20260812 06:58:50.844740 6130 rpc_server.cc:307] RPC server started. Bound to: 127.19.52.65:41283
I20260812 06:58:50.845006 6305 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.52.65:41283 every 8 connection(s)
I20260812 06:58:50.846015 6130 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517687384214-19665-0/minicluster-data/ts-0/data/info.pb
I20260812 06:58:50.853890 19665 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu as pid 6130
I20260812 06:58:50.854439 19665 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu
/tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517687384214-19665-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517687384214-19665-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517687384214-19665-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517687384214-19665-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.52.66:34423
--local_ip_for_outbound_sockets=127.19.52.66
--tserver_master_addrs=127.19.52.126:36401
--webserver_port=37613
--webserver_interface=127.19.52.66
--keytab_file=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517687384214-19665-0/krb5kdc/kudu_127.19.52.66.keytab
--principal=kudu/127.19.52.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.52.84:35215
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517687384214-19665-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517687384214-19665-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517687384214-19665-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 06:58:50.855067 6190 log.cc:824] T 63646b133f6c4739aa097080c3cf5ccf P aa4029c643b448ad9e197e92749c8fe6: Log is configured to *not* fsync() on all Append() calls
I20260812 06:58:50.863320 6190 tablet_bootstrap.cc:492] T 63646b133f6c4739aa097080c3cf5ccf P aa4029c643b448ad9e197e92749c8fe6: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260812 06:58:50.863801 6190 tablet_bootstrap.cc:492] T 63646b133f6c4739aa097080c3cf5ccf P aa4029c643b448ad9e197e92749c8fe6: Bootstrap complete.
I20260812 06:58:50.865525 6190 ts_tablet_manager.cc:1397] T 63646b133f6c4739aa097080c3cf5ccf P aa4029c643b448ad9e197e92749c8fe6: Time spent bootstrapping tablet: real 0.032s user 0.019s sys 0.011s
I20260812 06:58:50.869869 6190 raft_consensus.cc:348] T 63646b133f6c4739aa097080c3cf5ccf P aa4029c643b448ad9e197e92749c8fe6 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "666220fa0cdf403dad047d3011fed16f" member_type: VOTER last_known_addr { host: "127.19.52.67" port: 35465 } } peers { permanent_uuid: "25ef252218c24d6cae8e3f5efff644ea" member_type: VOTER last_known_addr { host: "127.19.52.66" port: 34423 } } peers { permanent_uuid: "aa4029c643b448ad9e197e92749c8fe6" member_type: VOTER last_known_addr { host: "127.19.52.65" port: 41283 } }
I20260812 06:58:50.870579 6190 raft_consensus.cc:729] T 63646b133f6c4739aa097080c3cf5ccf P aa4029c643b448ad9e197e92749c8fe6 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: aa4029c643b448ad9e197e92749c8fe6, State: Initialized, Role: FOLLOWER
I20260812 06:58:50.871161 6190 consensus_queue.cc:260] T 63646b133f6c4739aa097080c3cf5ccf P aa4029c643b448ad9e197e92749c8fe6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "666220fa0cdf403dad047d3011fed16f" member_type: VOTER last_known_addr { host: "127.19.52.67" port: 35465 } } peers { permanent_uuid: "25ef252218c24d6cae8e3f5efff644ea" member_type: VOTER last_known_addr { host: "127.19.52.66" port: 34423 } } peers { permanent_uuid: "aa4029c643b448ad9e197e92749c8fe6" member_type: VOTER last_known_addr { host: "127.19.52.65" port: 41283 } }
I20260812 06:58:50.872352 6190 ts_tablet_manager.cc:1428] T 63646b133f6c4739aa097080c3cf5ccf P aa4029c643b448ad9e197e92749c8fe6: Time spent starting tablet: real 0.007s user 0.006s sys 0.001s
W20260812 06:58:50.904012 6313 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 06:58:50.848439 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.126:36401 (local address 127.19.52.65:48215)
0812 06:58:50.849101 (+ 662us) negotiation.cc:107] Waiting for socket to connect
0812 06:58:50.849163 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0812 06:58:50.850204 (+ 1041us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:58:50.903033 (+ 52829us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.52.126:36401: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":389,"thread_start_us":161,"threads_started":1}
W20260812 06:58:50.904978 6307 heartbeater.cc:644] Failed to heartbeat to 127.19.52.126:36401 (0 consecutive failures): Network error: Failed to ping master at 127.19.52.126:36401: Client connection negotiation failed: client connection to 127.19.52.126:36401: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260812 06:58:50.905915 6313 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:58:50.905573 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.126:36401 (local address 127.19.52.65:59353)
0812 06:58:50.905738 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0812 06:58:50.905836 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.52.126:36401: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
I20260812 06:58:50.907055 6313 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:58:50.906778 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.126:36401 (local address 127.19.52.65:46887)
0812 06:58:50.906920 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0812 06:58:50.906977 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.52.126:36401: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
W20260812 06:58:50.907331 6307 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260812 06:58:50.981875 6315 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:58:50.982172 6315 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:58:50.982218 6315 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 06:58:50.986526 6315 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 06:58:50.986590 6315 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 06:58:50.986651 6315 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.52.66
I20260812 06:58:50.991945 6315 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.52.84:35215
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517687384214-19665-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517687384214-19665-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517687384214-19665-0/krb5kdc/kudu_127.19.52.66.keytab
--principal=kudu/127.19.52.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.52.66:34423
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517687384214-19665-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.19.52.66
--webserver_port=37613
--tserver_master_addrs=127.19.52.126:36401
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6315
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.52.66
--log_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517687384214-19665-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 06:15:54 UTC on ccycloud.quasar-rnfzce.root.comops.site
build id 4581989
I20260812 06:58:50.993019 6315 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:58:50.993969 6315 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 06:58:51.003504 6321 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:58:51.003476 6322 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:58:51.003492 6325 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:58:51.003780 6315 server_base.cc:1034] running on GCE node
I20260812 06:58:51.004276 6315 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:58:51.005003 6315 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:58:51.006201 6315 hybrid_clock.cc:648] HybridClock initialized: now 1786517931006182 us; error 30 us; skew 500 ppm
Aug 12 06:58:51 dist-test-slave-65mx krb5kdc[4794](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517931, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.52.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 06:58:51.009289 6315 init.cc:377] Logged in from keytab as kudu/127.19.52.66@KRBTEST.COM (short username kudu)
I20260812 06:58:51.010192 6315 webserver.cc:460] Webserver started at http://127.19.52.66:37613/ using document root <none> and password file <none>
I20260812 06:58:51.010816 6315 fs_manager.cc:359] Metadata directory not provided
I20260812 06:58:51.010911 6315 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:58:51.014837 6315 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260812 06:58:51.017206 6332 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 06:58:51.018523 6315 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 06:58:51.018676 6315 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517687384214-19665-0/minicluster-data/ts-1/data,/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517687384214-19665-0/minicluster-data/ts-1/wal
uuid: "25ef252218c24d6cae8e3f5efff644ea"
format_stamp: "Formatted at 2026-08-12 06:58:08 on dist-test-slave-65mx"
server_key: "8528faeac61ece0467e7f9d0be88de1a"
server_key_iv: "957fc3fb0af5270c72c4d5ba7c779a04"
server_key_version: "encryptionkey@0"
I20260812 06:58:51.019289 6315 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517687384214-19665-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517687384214-19665-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517687384214-19665-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:58:51.031044 6315 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 06:58:51.034490 6315 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 06:58:51.034716 6315 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:58:51.035346 6315 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 06:58:51.036262 6315 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 06:58:51.036332 6315 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 06:58:51.036402 6315 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 06:58:51.036454 6315 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 06:58:51.046195 6315 rpc_server.cc:307] RPC server started. Bound to: 127.19.52.66:34423
I20260812 06:58:51.046216 6460 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.52.66:34423 every 8 connection(s)
I20260812 06:58:51.047261 6315 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517687384214-19665-0/minicluster-data/ts-1/data/info.pb
I20260812 06:58:51.051180 6464 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:58:51.049421 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.126:36401 (local address 127.19.52.66:47921)
0812 06:58:51.049901 (+ 480us) negotiation.cc:107] Waiting for socket to connect
0812 06:58:51.050011 (+ 110us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.52.126:36401: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":278,"thread_start_us":117,"threads_started":1}
W20260812 06:58:51.051715 6461 heartbeater.cc:644] Failed to heartbeat to 127.19.52.126:36401 (0 consecutive failures): Network error: Failed to ping master at 127.19.52.126:36401: Client connection negotiation failed: client connection to 127.19.52.126:36401: connect: Connection refused (error 111)
I20260812 06:58:51.052469 6464 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:58:51.052234 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.126:36401 (local address 127.19.52.66:54655)
0812 06:58:51.052352 (+ 118us) negotiation.cc:107] Waiting for socket to connect
0812 06:58:51.052404 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.52.126:36401: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
I20260812 06:58:51.053117 6464 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:58:51.052909 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.126:36401 (local address 127.19.52.66:58395)
0812 06:58:51.053013 (+ 104us) negotiation.cc:107] Waiting for socket to connect
0812 06:58:51.053059 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.52.126:36401: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
W20260812 06:58:51.053308 6461 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260812 06:58:51.056792 19665 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu as pid 6315
I20260812 06:58:51.057279 19665 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu
/tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517687384214-19665-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517687384214-19665-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517687384214-19665-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517687384214-19665-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.52.67:35465
--local_ip_for_outbound_sockets=127.19.52.67
--tserver_master_addrs=127.19.52.126:36401
--webserver_port=41963
--webserver_interface=127.19.52.67
--keytab_file=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517687384214-19665-0/krb5kdc/kudu_127.19.52.67.keytab
--principal=kudu/127.19.52.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.52.84:35215
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517687384214-19665-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517687384214-19665-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517687384214-19665-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 06:58:51.170969 6465 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:58:51.171288 6465 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:58:51.171365 6465 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 06:58:51.175354 6465 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 06:58:51.175447 6465 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 06:58:51.175541 6465 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.52.67
I20260812 06:58:51.180840 6465 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.52.84:35215
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517687384214-19665-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517687384214-19665-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517687384214-19665-0/krb5kdc/kudu_127.19.52.67.keytab
--principal=kudu/127.19.52.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.52.67:35465
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517687384214-19665-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.19.52.67
--webserver_port=41963
--tserver_master_addrs=127.19.52.126:36401
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6465
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.52.67
--log_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517687384214-19665-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 06:15:54 UTC on ccycloud.quasar-rnfzce.root.comops.site
build id 4581989
I20260812 06:58:51.181950 6465 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:58:51.183024 6465 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 06:58:51.192777 6471 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:58:51.193018 6465 server_base.cc:1034] running on GCE node
W20260812 06:58:51.192780 6474 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:58:51.192778 6472 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:58:51.193529 6465 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:58:51.194152 6465 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:58:51.195338 6465 hybrid_clock.cc:648] HybridClock initialized: now 1786517931195318 us; error 30 us; skew 500 ppm
Aug 12 06:58:51 dist-test-slave-65mx krb5kdc[4794](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517931, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.52.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 06:58:51.198225 6465 init.cc:377] Logged in from keytab as kudu/127.19.52.67@KRBTEST.COM (short username kudu)
I20260812 06:58:51.199218 6465 webserver.cc:460] Webserver started at http://127.19.52.67:41963/ using document root <none> and password file <none>
I20260812 06:58:51.199879 6465 fs_manager.cc:359] Metadata directory not provided
I20260812 06:58:51.200047 6465 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:58:51.204306 6465 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.000s
I20260812 06:58:51.206738 6481 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 06:58:51.208338 6465 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260812 06:58:51.208503 6465 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517687384214-19665-0/minicluster-data/ts-2/data,/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517687384214-19665-0/minicluster-data/ts-2/wal
uuid: "666220fa0cdf403dad047d3011fed16f"
format_stamp: "Formatted at 2026-08-12 06:58:08 on dist-test-slave-65mx"
server_key: "76e41badd775cd19f48c41252a471152"
server_key_iv: "8b7221af01d65e7fcbc265f760f1f591"
server_key_version: "encryptionkey@0"
I20260812 06:58:51.209147 6465 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517687384214-19665-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517687384214-19665-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517687384214-19665-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:58:51.225479 6465 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 06:58:51.228912 6465 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 06:58:51.229135 6465 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:58:51.229774 6465 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 06:58:51.230650 6465 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 06:58:51.230724 6465 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 06:58:51.230804 6465 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 06:58:51.230849 6465 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 06:58:51.240268 6465 rpc_server.cc:307] RPC server started. Bound to: 127.19.52.67:35465
I20260812 06:58:51.240306 6610 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.52.67:35465 every 8 connection(s)
I20260812 06:58:51.241319 6465 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517687384214-19665-0/minicluster-data/ts-2/data/info.pb
I20260812 06:58:51.245221 19665 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu as pid 6465
I20260812 06:58:51.245512 6613 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:58:51.243640 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.126:36401 (local address 127.19.52.67:59585)
0812 06:58:51.244145 (+ 505us) negotiation.cc:107] Waiting for socket to connect
0812 06:58:51.244251 (+ 106us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.52.126:36401: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":305,"thread_start_us":135,"threads_started":1}
W20260812 06:58:51.245994 6611 heartbeater.cc:644] Failed to heartbeat to 127.19.52.126:36401 (0 consecutive failures): Network error: Failed to ping master at 127.19.52.126:36401: Client connection negotiation failed: client connection to 127.19.52.126:36401: connect: Connection refused (error 111)
I20260812 06:58:51.246702 6613 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:58:51.246405 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.126:36401 (local address 127.19.52.67:52155)
0812 06:58:51.246558 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0812 06:58:51.246621 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.52.126:36401: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
I20260812 06:58:51.247388 6613 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:58:51.247135 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.126:36401 (local address 127.19.52.67:45657)
0812 06:58:51.247264 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0812 06:58:51.247317 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.52.126:36401: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
W20260812 06:58:51.247623 6611 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260812 06:58:51.908634 6615 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:58:51.908062 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.126:36401 (local address 127.19.52.65:33793)
0812 06:58:51.908383 (+ 321us) negotiation.cc:107] Waiting for socket to connect
0812 06:58:51.908451 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.52.126:36401: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":231,"thread_start_us":95,"threads_started":1}
I20260812 06:58:52.054352 6616 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:58:52.053880 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.126:36401 (local address 127.19.52.66:35801)
0812 06:58:52.054178 (+ 298us) negotiation.cc:107] Waiting for socket to connect
0812 06:58:52.054239 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.52.126:36401: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":198,"thread_start_us":86,"threads_started":1}
I20260812 06:58:52.248736 6617 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:58:52.248281 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.126:36401 (local address 127.19.52.67:33423)
0812 06:58:52.248528 (+ 247us) negotiation.cc:107] Waiting for socket to connect
0812 06:58:52.248632 (+ 104us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.52.126:36401: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":179,"thread_start_us":72,"threads_started":1}
I20260812 06:58:52.325767 6620 raft_consensus.cc:482] T 63646b133f6c4739aa097080c3cf5ccf P aa4029c643b448ad9e197e92749c8fe6 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 06:58:52.326052 6620 raft_consensus.cc:504] T 63646b133f6c4739aa097080c3cf5ccf P aa4029c643b448ad9e197e92749c8fe6 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "666220fa0cdf403dad047d3011fed16f" member_type: VOTER last_known_addr { host: "127.19.52.67" port: 35465 } } peers { permanent_uuid: "25ef252218c24d6cae8e3f5efff644ea" member_type: VOTER last_known_addr { host: "127.19.52.66" port: 34423 } } peers { permanent_uuid: "aa4029c643b448ad9e197e92749c8fe6" member_type: VOTER last_known_addr { host: "127.19.52.65" port: 41283 } }
I20260812 06:58:52.327188 6620 leader_election.cc:288] T 63646b133f6c4739aa097080c3cf5ccf P aa4029c643b448ad9e197e92749c8fe6 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 666220fa0cdf403dad047d3011fed16f (127.19.52.67:35465), 25ef252218c24d6cae8e3f5efff644ea (127.19.52.66:34423)
Aug 12 06:58:52 dist-test-slave-65mx krb5kdc[4794](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517930, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.52.65@KRBTEST.COM for kudu/127.19.52.67@KRBTEST.COM
Aug 12 06:58:52 dist-test-slave-65mx krb5kdc[4794](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517930, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.52.65@KRBTEST.COM for kudu/127.19.52.66@KRBTEST.COM
I20260812 06:58:52.342813 6615 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:58:52.327416 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.67:35465 (local address 127.19.52.65:52285)
0812 06:58:52.327548 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0812 06:58:52.327566 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0812 06:58:52.327686 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:58:52.329578 (+ 1892us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:58:52.329584 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:58:52.330207 (+ 623us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:58:52.330683 (+ 476us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:58:52.330698 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:58:52.331999 (+ 1301us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:58:52.332002 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:58:52.333012 (+ 1010us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:58:52.333022 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:58:52.333864 (+ 842us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:58:52.333868 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:58:52.334013 (+ 145us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:58:52.334757 (+ 744us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 06:58:52.334769 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:58:52.336836 (+ 2067us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 06:58:52.339599 (+ 2763us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:58:52.339618 (+ 19us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:58:52.339621 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:58:52.340133 (+ 512us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:58:52.340574 (+ 441us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:58:52.340577 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:58:52.340579 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:58:52.340787 (+ 208us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:58:52.341580 (+ 793us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 06:58:52.341587 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 06:58:52.341988 (+ 401us) client_negotiation.cc:769] Sending connection context
0812 06:58:52.342204 (+ 216us) client_negotiation.cc:240] Negotiation successful
0812 06:58:52.342659 (+ 455us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":50,"mutex_wait_us":333,"spinlock_wait_cycles":60416}
I20260812 06:58:52.342866 6622 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:58:52.327615 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.52.65:52285 (local address 127.19.52.67:35465)
0812 06:58:52.327916 (+ 301us) server_negotiation.cc:206] Beginning negotiation
0812 06:58:52.327943 (+ 27us) server_negotiation.cc:399] Waiting for connection header
0812 06:58:52.327991 (+ 48us) server_negotiation.cc:407] Connection header received
0812 06:58:52.328843 (+ 852us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:58:52.328850 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:58:52.329208 (+ 358us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:58:52.329493 (+ 285us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:58:52.330876 (+ 1383us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:58:52.331850 (+ 974us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:58:52.333144 (+ 1294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:58:52.333697 (+ 553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:58:52.333900 (+ 203us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:58:52.337114 (+ 3214us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:58:52.337122 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:58:52.337140 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:58:52.337176 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:58:52.339451 (+ 2275us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:58:52.340272 (+ 821us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:58:52.340279 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:58:52.340284 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:58:52.340459 (+ 175us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:58:52.340898 (+ 439us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:58:52.340901 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:58:52.340903 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:58:52.341465 (+ 562us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:58:52.341592 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0812 06:58:52.342373 (+ 781us) server_negotiation.cc:299] Negotiation successful
0812 06:58:52.342675 (+ 302us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":190,"thread_start_us":114,"threads_started":1}
I20260812 06:58:52.342813 6624 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:58:52.327438 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.66:34423 (local address 127.19.52.65:44069)
0812 06:58:52.328048 (+ 610us) negotiation.cc:107] Waiting for socket to connect
0812 06:58:52.328062 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 06:58:52.328184 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:58:52.329578 (+ 1394us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:58:52.329584 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:58:52.330203 (+ 619us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:58:52.330683 (+ 480us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:58:52.330698 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:58:52.332000 (+ 1302us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:58:52.332003 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:58:52.333037 (+ 1034us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:58:52.333048 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:58:52.333798 (+ 750us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:58:52.333805 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:58:52.334012 (+ 207us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:58:52.334711 (+ 699us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 06:58:52.334729 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:58:52.337044 (+ 2315us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 06:58:52.339599 (+ 2555us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:58:52.339606 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:58:52.339621 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:58:52.340133 (+ 512us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:58:52.340574 (+ 441us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:58:52.340577 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:58:52.340579 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:58:52.340787 (+ 208us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:58:52.341580 (+ 793us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 06:58:52.341587 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 06:58:52.341988 (+ 401us) client_negotiation.cc:769] Sending connection context
0812 06:58:52.342204 (+ 216us) client_negotiation.cc:240] Negotiation successful
0812 06:58:52.342598 (+ 394us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":542,"mutex_wait_us":558,"spinlock_wait_cycles":45184,"thread_start_us":74,"threads_started":1}
I20260812 06:58:52.342871 6623 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:58:52.327635 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.52.65:44069 (local address 127.19.52.66:34423)
0812 06:58:52.328061 (+ 426us) server_negotiation.cc:206] Beginning negotiation
0812 06:58:52.328085 (+ 24us) server_negotiation.cc:399] Waiting for connection header
0812 06:58:52.328170 (+ 85us) server_negotiation.cc:407] Connection header received
0812 06:58:52.328962 (+ 792us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:58:52.328968 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:58:52.329247 (+ 279us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:58:52.329542 (+ 295us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:58:52.330888 (+ 1346us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:58:52.331845 (+ 957us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:58:52.333195 (+ 1350us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:58:52.333660 (+ 465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:58:52.333815 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:58:52.337214 (+ 3399us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:58:52.337220 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:58:52.337232 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:58:52.337280 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:58:52.339441 (+ 2161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:58:52.340275 (+ 834us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:58:52.340280 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:58:52.340284 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:58:52.340436 (+ 152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:58:52.340898 (+ 462us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:58:52.340901 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:58:52.340903 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:58:52.341445 (+ 542us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:58:52.341592 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0812 06:58:52.342373 (+ 781us) server_negotiation.cc:299] Negotiation successful
0812 06:58:52.342675 (+ 302us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":322,"thread_start_us":99,"threads_started":1}
W20260812 06:58:52.345549 6186 leader_election.cc:341] T 63646b133f6c4739aa097080c3cf5ccf P aa4029c643b448ad9e197e92749c8fe6 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 25ef252218c24d6cae8e3f5efff644ea (127.19.52.66:34423): Not found: Tablet not found: 63646b133f6c4739aa097080c3cf5ccf
W20260812 06:58:52.345722 6185 leader_election.cc:341] T 63646b133f6c4739aa097080c3cf5ccf P aa4029c643b448ad9e197e92749c8fe6 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 666220fa0cdf403dad047d3011fed16f (127.19.52.67:35465): Not found: Tablet not found: 63646b133f6c4739aa097080c3cf5ccf
I20260812 06:58:52.345806 6185 leader_election.cc:302] T 63646b133f6c4739aa097080c3cf5ccf P aa4029c643b448ad9e197e92749c8fe6 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aa4029c643b448ad9e197e92749c8fe6; no voters: 25ef252218c24d6cae8e3f5efff644ea, 666220fa0cdf403dad047d3011fed16f
I20260812 06:58:52.346035 6620 raft_consensus.cc:2726] T 63646b133f6c4739aa097080c3cf5ccf P aa4029c643b448ad9e197e92749c8fe6 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260812 06:58:52.909976 6626 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:58:52.909504 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.126:36401 (local address 127.19.52.65:42699)
0812 06:58:52.909814 (+ 310us) negotiation.cc:107] Waiting for socket to connect
0812 06:58:52.909875 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.52.126:36401: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":217,"thread_start_us":126,"threads_started":1}
I20260812 06:58:53.055514 6627 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:58:53.055115 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.126:36401 (local address 127.19.52.66:50381)
0812 06:58:53.055367 (+ 252us) negotiation.cc:107] Waiting for socket to connect
0812 06:58:53.055423 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.52.126:36401: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":180,"thread_start_us":71,"threads_started":1}
I20260812 06:58:53.250090 6628 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:58:53.249595 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.126:36401 (local address 127.19.52.67:40695)
0812 06:58:53.249907 (+ 312us) negotiation.cc:107] Waiting for socket to connect
0812 06:58:53.249986 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.52.126:36401: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":229,"thread_start_us":113,"threads_started":1}
I20260812 06:58:53.911522 6629 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:58:53.911046 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.126:36401 (local address 127.19.52.65:48835)
0812 06:58:53.911349 (+ 303us) negotiation.cc:107] Waiting for socket to connect
0812 06:58:53.911418 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.52.126:36401: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":215,"thread_start_us":83,"threads_started":1}
I20260812 06:58:54.057090 6630 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:58:54.056373 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.126:36401 (local address 127.19.52.66:37339)
0812 06:58:54.056916 (+ 543us) negotiation.cc:107] Waiting for socket to connect
0812 06:58:54.056992 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.52.126:36401: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":460,"thread_start_us":122,"threads_started":1}
I20260812 06:58:54.107523 6631 raft_consensus.cc:482] T 63646b133f6c4739aa097080c3cf5ccf P aa4029c643b448ad9e197e92749c8fe6 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 06:58:54.107702 6631 raft_consensus.cc:504] T 63646b133f6c4739aa097080c3cf5ccf P aa4029c643b448ad9e197e92749c8fe6 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "666220fa0cdf403dad047d3011fed16f" member_type: VOTER last_known_addr { host: "127.19.52.67" port: 35465 } } peers { permanent_uuid: "25ef252218c24d6cae8e3f5efff644ea" member_type: VOTER last_known_addr { host: "127.19.52.66" port: 34423 } } peers { permanent_uuid: "aa4029c643b448ad9e197e92749c8fe6" member_type: VOTER last_known_addr { host: "127.19.52.65" port: 41283 } }
I20260812 06:58:54.108198 6631 leader_election.cc:288] T 63646b133f6c4739aa097080c3cf5ccf P aa4029c643b448ad9e197e92749c8fe6 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 666220fa0cdf403dad047d3011fed16f (127.19.52.67:35465), 25ef252218c24d6cae8e3f5efff644ea (127.19.52.66:34423)
W20260812 06:58:54.109118 6186 leader_election.cc:341] T 63646b133f6c4739aa097080c3cf5ccf P aa4029c643b448ad9e197e92749c8fe6 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 25ef252218c24d6cae8e3f5efff644ea (127.19.52.66:34423): Not found: Tablet not found: 63646b133f6c4739aa097080c3cf5ccf
W20260812 06:58:54.109217 6185 leader_election.cc:341] T 63646b133f6c4739aa097080c3cf5ccf P aa4029c643b448ad9e197e92749c8fe6 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 666220fa0cdf403dad047d3011fed16f (127.19.52.67:35465): Not found: Tablet not found: 63646b133f6c4739aa097080c3cf5ccf
I20260812 06:58:54.109278 6185 leader_election.cc:302] T 63646b133f6c4739aa097080c3cf5ccf P aa4029c643b448ad9e197e92749c8fe6 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aa4029c643b448ad9e197e92749c8fe6; no voters: 25ef252218c24d6cae8e3f5efff644ea, 666220fa0cdf403dad047d3011fed16f
I20260812 06:58:54.109431 6631 raft_consensus.cc:2726] T 63646b133f6c4739aa097080c3cf5ccf P aa4029c643b448ad9e197e92749c8fe6 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260812 06:58:54.251358 6632 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:58:54.250900 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.126:36401 (local address 127.19.52.67:34717)
0812 06:58:54.251200 (+ 300us) negotiation.cc:107] Waiting for socket to connect
0812 06:58:54.251260 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.52.126:36401: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":205,"thread_start_us":85,"threads_started":1}
I20260812 06:58:54.913358 6633 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:58:54.912770 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.126:36401 (local address 127.19.52.65:53045)
0812 06:58:54.913041 (+ 271us) negotiation.cc:107] Waiting for socket to connect
0812 06:58:54.913253 (+ 212us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.52.126:36401: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":193,"thread_start_us":100,"threads_started":1}
I20260812 06:58:55.058269 6634 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:58:55.057882 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.126:36401 (local address 127.19.52.66:59357)
0812 06:58:55.058127 (+ 245us) negotiation.cc:107] Waiting for socket to connect
0812 06:58:55.058180 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.52.126:36401: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":174,"thread_start_us":73,"threads_started":1}
I20260812 06:58:55.252877 6635 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:58:55.252364 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.126:36401 (local address 127.19.52.67:38057)
0812 06:58:55.252707 (+ 343us) negotiation.cc:107] Waiting for socket to connect
0812 06:58:55.252776 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.52.126:36401: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":264,"thread_start_us":98,"threads_started":1}
I20260812 06:58:55.914798 6637 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:58:55.914286 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.126:36401 (local address 127.19.52.65:48045)
0812 06:58:55.914619 (+ 333us) negotiation.cc:107] Waiting for socket to connect
0812 06:58:55.914681 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.52.126:36401: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":257,"thread_start_us":106,"threads_started":1}
I20260812 06:58:56.059999 6638 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:58:56.059366 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.126:36401 (local address 127.19.52.66:53999)
0812 06:58:56.059755 (+ 389us) negotiation.cc:107] Waiting for socket to connect
0812 06:58:56.059846 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.52.126:36401: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":271,"thread_start_us":140,"threads_started":1}
I20260812 06:58:56.254575 6639 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:58:56.253885 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.126:36401 (local address 127.19.52.67:58023)
0812 06:58:56.254293 (+ 408us) negotiation.cc:107] Waiting for socket to connect
0812 06:58:56.254364 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.52.126:36401: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":319,"thread_start_us":114,"threads_started":1}
I20260812 06:58:56.916201 6640 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:58:56.915704 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.126:36401 (local address 127.19.52.65:34205)
0812 06:58:56.916038 (+ 334us) negotiation.cc:107] Waiting for socket to connect
0812 06:58:56.916092 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.52.126:36401: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":253,"thread_start_us":112,"threads_started":1}
I20260812 06:58:56.922132 6641 raft_consensus.cc:482] T 63646b133f6c4739aa097080c3cf5ccf P aa4029c643b448ad9e197e92749c8fe6 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 06:58:56.922330 6641 raft_consensus.cc:504] T 63646b133f6c4739aa097080c3cf5ccf P aa4029c643b448ad9e197e92749c8fe6 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "666220fa0cdf403dad047d3011fed16f" member_type: VOTER last_known_addr { host: "127.19.52.67" port: 35465 } } peers { permanent_uuid: "25ef252218c24d6cae8e3f5efff644ea" member_type: VOTER last_known_addr { host: "127.19.52.66" port: 34423 } } peers { permanent_uuid: "aa4029c643b448ad9e197e92749c8fe6" member_type: VOTER last_known_addr { host: "127.19.52.65" port: 41283 } }
I20260812 06:58:56.922923 6641 leader_election.cc:288] T 63646b133f6c4739aa097080c3cf5ccf P aa4029c643b448ad9e197e92749c8fe6 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 666220fa0cdf403dad047d3011fed16f (127.19.52.67:35465), 25ef252218c24d6cae8e3f5efff644ea (127.19.52.66:34423)
W20260812 06:58:56.923780 6186 leader_election.cc:341] T 63646b133f6c4739aa097080c3cf5ccf P aa4029c643b448ad9e197e92749c8fe6 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 25ef252218c24d6cae8e3f5efff644ea (127.19.52.66:34423): Not found: Tablet not found: 63646b133f6c4739aa097080c3cf5ccf
W20260812 06:58:56.923923 6185 leader_election.cc:341] T 63646b133f6c4739aa097080c3cf5ccf P aa4029c643b448ad9e197e92749c8fe6 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 666220fa0cdf403dad047d3011fed16f (127.19.52.67:35465): Not found: Tablet not found: 63646b133f6c4739aa097080c3cf5ccf
I20260812 06:58:56.924005 6185 leader_election.cc:302] T 63646b133f6c4739aa097080c3cf5ccf P aa4029c643b448ad9e197e92749c8fe6 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aa4029c643b448ad9e197e92749c8fe6; no voters: 25ef252218c24d6cae8e3f5efff644ea, 666220fa0cdf403dad047d3011fed16f
I20260812 06:58:56.924188 6641 raft_consensus.cc:2726] T 63646b133f6c4739aa097080c3cf5ccf P aa4029c643b448ad9e197e92749c8fe6 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260812 06:58:57.061476 6642 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:58:57.061037 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.126:36401 (local address 127.19.52.66:56361)
0812 06:58:57.061308 (+ 271us) negotiation.cc:107] Waiting for socket to connect
0812 06:58:57.061366 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.52.126:36401: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":197,"thread_start_us":81,"threads_started":1}
I20260812 06:58:57.255983 6643 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:58:57.255504 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.126:36401 (local address 127.19.52.67:49069)
0812 06:58:57.255815 (+ 311us) negotiation.cc:107] Waiting for socket to connect
0812 06:58:57.255880 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.52.126:36401: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":233,"thread_start_us":99,"threads_started":1}
I20260812 06:58:57.917915 6644 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:58:57.917296 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.126:36401 (local address 127.19.52.65:48583)
0812 06:58:57.917708 (+ 412us) negotiation.cc:107] Waiting for socket to connect
0812 06:58:57.917786 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.52.126:36401: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":283,"thread_start_us":99,"threads_started":1}
I20260812 06:58:58.063169 6645 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:58:58.062579 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.126:36401 (local address 127.19.52.66:41335)
0812 06:58:58.062949 (+ 370us) negotiation.cc:107] Waiting for socket to connect
0812 06:58:58.063040 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.52.126:36401: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":278,"thread_start_us":125,"threads_started":1}
I20260812 06:58:58.257606 6646 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:58:58.257059 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.126:36401 (local address 127.19.52.67:49823)
0812 06:58:58.257429 (+ 370us) negotiation.cc:107] Waiting for socket to connect
0812 06:58:58.257504 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.52.126:36401: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":286,"thread_start_us":88,"threads_started":1}
I20260812 06:58:58.919292 6647 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:58:58.918796 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.126:36401 (local address 127.19.52.65:41397)
0812 06:58:58.919109 (+ 313us) negotiation.cc:107] Waiting for socket to connect
0812 06:58:58.919181 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.52.126:36401: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":220,"thread_start_us":86,"threads_started":1}
I20260812 06:58:59.064653 6648 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:58:59.064075 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.126:36401 (local address 127.19.52.66:46871)
0812 06:58:59.064410 (+ 335us) negotiation.cc:107] Waiting for socket to connect
0812 06:58:59.064478 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.52.126:36401: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":246,"thread_start_us":104,"threads_started":1}
I20260812 06:58:59.258976 6649 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:58:59.258523 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.126:36401 (local address 127.19.52.67:40479)
0812 06:58:59.258804 (+ 281us) negotiation.cc:107] Waiting for socket to connect
0812 06:58:59.258870 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.52.126:36401: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":201,"thread_start_us":84,"threads_started":1}
I20260812 06:58:59.920922 6651 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:58:59.920340 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.126:36401 (local address 127.19.52.65:53375)
0812 06:58:59.920750 (+ 410us) negotiation.cc:107] Waiting for socket to connect
0812 06:58:59.920820 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.52.126:36401: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":321,"thread_start_us":83,"threads_started":1}
I20260812 06:59:00.066097 6652 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:59:00.065548 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.126:36401 (local address 127.19.52.66:57059)
0812 06:59:00.065911 (+ 363us) negotiation.cc:107] Waiting for socket to connect
0812 06:59:00.065992 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.52.126:36401: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":270,"thread_start_us":119,"threads_started":1}
I20260812 06:59:00.260370 6653 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:59:00.259885 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.126:36401 (local address 127.19.52.67:50911)
0812 06:59:00.260199 (+ 314us) negotiation.cc:107] Waiting for socket to connect
0812 06:59:00.260265 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.52.126:36401: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":229,"thread_start_us":84,"threads_started":1}
I20260812 06:59:00.922442 6655 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:59:00.921989 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.126:36401 (local address 127.19.52.65:59039)
0812 06:59:00.922269 (+ 280us) negotiation.cc:107] Waiting for socket to connect
0812 06:59:00.922337 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.52.126:36401: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":192,"thread_start_us":80,"threads_started":1}
I20260812 06:59:01.067720 6656 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:59:01.067185 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.126:36401 (local address 127.19.52.66:45807)
0812 06:59:01.067541 (+ 356us) negotiation.cc:107] Waiting for socket to connect
0812 06:59:01.067615 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.52.126:36401: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":255,"thread_start_us":104,"threads_started":1}
I20260812 06:59:01.262075 6657 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:59:01.261584 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.126:36401 (local address 127.19.52.67:46135)
0812 06:59:01.261882 (+ 298us) negotiation.cc:107] Waiting for socket to connect
0812 06:59:01.261945 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.52.126:36401: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":217,"thread_start_us":97,"threads_started":1}
I20260812 06:59:01.763291 6658 raft_consensus.cc:482] T 63646b133f6c4739aa097080c3cf5ccf P aa4029c643b448ad9e197e92749c8fe6 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 06:59:01.763430 6658 raft_consensus.cc:504] T 63646b133f6c4739aa097080c3cf5ccf P aa4029c643b448ad9e197e92749c8fe6 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "666220fa0cdf403dad047d3011fed16f" member_type: VOTER last_known_addr { host: "127.19.52.67" port: 35465 } } peers { permanent_uuid: "25ef252218c24d6cae8e3f5efff644ea" member_type: VOTER last_known_addr { host: "127.19.52.66" port: 34423 } } peers { permanent_uuid: "aa4029c643b448ad9e197e92749c8fe6" member_type: VOTER last_known_addr { host: "127.19.52.65" port: 41283 } }
I20260812 06:59:01.763980 6658 leader_election.cc:288] T 63646b133f6c4739aa097080c3cf5ccf P aa4029c643b448ad9e197e92749c8fe6 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 666220fa0cdf403dad047d3011fed16f (127.19.52.67:35465), 25ef252218c24d6cae8e3f5efff644ea (127.19.52.66:34423)
W20260812 06:59:01.765000 6185 leader_election.cc:341] T 63646b133f6c4739aa097080c3cf5ccf P aa4029c643b448ad9e197e92749c8fe6 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 666220fa0cdf403dad047d3011fed16f (127.19.52.67:35465): Not found: Tablet not found: 63646b133f6c4739aa097080c3cf5ccf
W20260812 06:59:01.765139 6186 leader_election.cc:341] T 63646b133f6c4739aa097080c3cf5ccf P aa4029c643b448ad9e197e92749c8fe6 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 25ef252218c24d6cae8e3f5efff644ea (127.19.52.66:34423): Not found: Tablet not found: 63646b133f6c4739aa097080c3cf5ccf
I20260812 06:59:01.765232 6186 leader_election.cc:302] T 63646b133f6c4739aa097080c3cf5ccf P aa4029c643b448ad9e197e92749c8fe6 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aa4029c643b448ad9e197e92749c8fe6; no voters: 25ef252218c24d6cae8e3f5efff644ea, 666220fa0cdf403dad047d3011fed16f
I20260812 06:59:01.765431 6658 raft_consensus.cc:2726] T 63646b133f6c4739aa097080c3cf5ccf P aa4029c643b448ad9e197e92749c8fe6 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260812 06:59:01.923862 6659 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:59:01.923317 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.126:36401 (local address 127.19.52.65:47187)
0812 06:59:01.923647 (+ 330us) negotiation.cc:107] Waiting for socket to connect
0812 06:59:01.923729 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.52.126:36401: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":213,"thread_start_us":91,"threads_started":1}
I20260812 06:59:02.069063 6660 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:59:02.068612 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.126:36401 (local address 127.19.52.66:43791)
0812 06:59:02.068911 (+ 299us) negotiation.cc:107] Waiting for socket to connect
0812 06:59:02.068966 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.52.126:36401: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":213,"thread_start_us":105,"threads_started":1}
I20260812 06:59:02.263550 6661 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:59:02.263068 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.126:36401 (local address 127.19.52.67:37707)
0812 06:59:02.263374 (+ 306us) negotiation.cc:107] Waiting for socket to connect
0812 06:59:02.263443 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.52.126:36401: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":223,"thread_start_us":96,"threads_started":1}
I20260812 06:59:02.925235 6662 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:59:02.924783 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.126:36401 (local address 127.19.52.65:43769)
0812 06:59:02.925051 (+ 268us) negotiation.cc:107] Waiting for socket to connect
0812 06:59:02.925125 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.52.126:36401: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":189,"thread_start_us":76,"threads_started":1}
I20260812 06:59:03.070364 6663 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:59:03.069907 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.126:36401 (local address 127.19.52.66:32911)
0812 06:59:03.070204 (+ 297us) negotiation.cc:107] Waiting for socket to connect
0812 06:59:03.070264 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.52.126:36401: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":199,"thread_start_us":109,"threads_started":1}
I20260812 06:59:03.265100 6664 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:59:03.264511 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.126:36401 (local address 127.19.52.67:42473)
0812 06:59:03.264860 (+ 349us) negotiation.cc:107] Waiting for socket to connect
0812 06:59:03.264930 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.52.126:36401: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":265,"thread_start_us":80,"threads_started":1}
I20260812 06:59:03.926616 6665 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:59:03.926147 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.126:36401 (local address 127.19.52.65:56511)
0812 06:59:03.926441 (+ 294us) negotiation.cc:107] Waiting for socket to connect
0812 06:59:03.926512 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.52.126:36401: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":204,"thread_start_us":81,"threads_started":1}
I20260812 06:59:04.071688 6666 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:59:04.071199 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.126:36401 (local address 127.19.52.66:60269)
0812 06:59:04.071521 (+ 322us) negotiation.cc:107] Waiting for socket to connect
0812 06:59:04.071584 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.52.126:36401: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":224,"thread_start_us":82,"threads_started":1}
I20260812 06:59:04.266587 6669 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:59:04.266088 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.126:36401 (local address 127.19.52.67:54563)
0812 06:59:04.266407 (+ 319us) negotiation.cc:107] Waiting for socket to connect
0812 06:59:04.266466 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.52.126:36401: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":243,"thread_start_us":116,"threads_started":1}
I20260812 06:59:04.928010 6670 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:59:04.927566 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.126:36401 (local address 127.19.52.65:47019)
0812 06:59:04.927853 (+ 287us) negotiation.cc:107] Waiting for socket to connect
0812 06:59:04.927914 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.52.126:36401: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":204,"thread_start_us":80,"threads_started":1}
I20260812 06:59:05.073172 6671 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:59:05.072680 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.126:36401 (local address 127.19.52.66:52401)
0812 06:59:05.073003 (+ 323us) negotiation.cc:107] Waiting for socket to connect
0812 06:59:05.073069 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.52.126:36401: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":229,"thread_start_us":103,"threads_started":1}
I20260812 06:59:05.267966 6672 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:59:05.267514 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.126:36401 (local address 127.19.52.67:57205)
0812 06:59:05.267794 (+ 280us) negotiation.cc:107] Waiting for socket to connect
0812 06:59:05.267859 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.52.126:36401: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":197,"thread_start_us":92,"threads_started":1}
I20260812 06:59:05.929908 6673 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:59:05.929280 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.126:36401 (local address 127.19.52.65:35801)
0812 06:59:05.929642 (+ 362us) negotiation.cc:107] Waiting for socket to connect
0812 06:59:05.929766 (+ 124us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.52.126:36401: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":249,"thread_start_us":75,"threads_started":1}
I20260812 06:59:06.074700 6674 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:59:06.074204 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.126:36401 (local address 127.19.52.66:37733)
0812 06:59:06.074523 (+ 319us) negotiation.cc:107] Waiting for socket to connect
0812 06:59:06.074598 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.52.126:36401: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":221,"thread_start_us":79,"threads_started":1}
I20260812 06:59:06.247422 19665 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu with pid 6130
I20260812 06:59:06.256251 19665 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu with pid 6315
I20260812 06:59:06.264474 19665 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu with pid 6465
2026-08-12T06:59:06Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedTSK (59637 ms)
[----------] 30 tests from SecurityITest (258842 ms total)
[----------] 6 tests from EncryptionPolicyTest
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/0
Loading random data
Initializing database '/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786517687384214-19665-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 06:59:06 dist-test-slave-65mx krb5kdc[6678](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 06:59:06 dist-test-slave-65mx krb5kdc[6678](info): set up 2 sockets
Aug 12 06:59:06 dist-test-slave-65mx krb5kdc[6678](info): commencing operation
krb5kdc: starting...
W20260812 06:59:07.407651 19665 mini_kdc.cc:121] Time spent starting KDC: real 1.118s user 0.002s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786517687384214-19665-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786517687384214-19665-0/krb5kdc/test-user.keytab.
Aug 12 06:59:07 dist-test-slave-65mx krb5kdc[6678](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517947, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T06:59:07Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T06:59:07Z Disabled control of system clock
WARNING: no policy specified for kudu/127.19.52.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.52.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.52.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786517687384214-19665-0/krb5kdc/kudu_127.19.52.126.keytab.
Entry for principal kudu/127.19.52.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786517687384214-19665-0/krb5kdc/kudu_127.19.52.126.keytab.
I20260812 06:59:07.557396 19665 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu
/tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786517687384214-19665-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786517687384214-19665-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786517687384214-19665-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786517687384214-19665-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.52.126:44425
--webserver_interface=127.19.52.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786517687384214-19665-0/krb5kdc/kudu_127.19.52.126.keytab
--principal=kudu/127.19.52.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.52.84:46593
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.19.52.126:44425
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786517687384214-19665-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786517687384214-19665-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786517687384214-19665-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 06:59:07.678936 6692 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:59:07.679247 6692 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:59:07.679292 6692 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 06:59:07.683377 6692 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 06:59:07.683449 6692 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 06:59:07.683470 6692 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 06:59:07.683486 6692 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 06:59:07.683502 6692 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 06:59:07.689152 6692 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.52.84:46593
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786517687384214-19665-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786517687384214-19665-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.19.52.126:44425
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786517687384214-19665-0/krb5kdc/kudu_127.19.52.126.keytab
--principal=kudu/127.19.52.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.52.126:44425
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786517687384214-19665-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.19.52.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6692
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786517687384214-19665-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 06:15:54 UTC on ccycloud.quasar-rnfzce.root.comops.site
build id 4581989
I20260812 06:59:07.690359 6692 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:59:07.691358 6692 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 06:59:07.701015 6697 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:59:07.701006 6698 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:59:07.701058 6692 server_base.cc:1034] running on GCE node
W20260812 06:59:07.700990 6700 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:59:07.702004 6692 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:59:07.703172 6692 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:59:07.704370 6692 hybrid_clock.cc:648] HybridClock initialized: now 1786517947704367 us; error 49 us; skew 500 ppm
Aug 12 06:59:07 dist-test-slave-65mx krb5kdc[6678](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517947, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.52.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 06:59:07.713384 6692 init.cc:377] Logged in from keytab as kudu/127.19.52.126@KRBTEST.COM (short username kudu)
I20260812 06:59:07.714433 6692 webserver.cc:460] Webserver started at http://127.19.52.126:43329/ using document root <none> and password file <none>
I20260812 06:59:07.715054 6692 fs_manager.cc:359] Metadata directory not provided
I20260812 06:59:07.715160 6692 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:59:07.715404 6692 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 06:59:07.717710 6692 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786517687384214-19665-0/minicluster-data/master-0/data/instance:
uuid: "8801dcfd547a41afb3b537a64694b310"
format_stamp: "Formatted at 2026-08-12 06:59:07 on dist-test-slave-65mx"
server_key: "8f716b56fbcad317ed0511fecb5eb87c"
server_key_iv: "5024ee8d91efca6be3557a53632fcad7"
server_key_version: "encryptionkey@0"
I20260812 06:59:07.718304 6692 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786517687384214-19665-0/minicluster-data/master-0/wal/instance:
uuid: "8801dcfd547a41afb3b537a64694b310"
format_stamp: "Formatted at 2026-08-12 06:59:07 on dist-test-slave-65mx"
server_key: "8f716b56fbcad317ed0511fecb5eb87c"
server_key_iv: "5024ee8d91efca6be3557a53632fcad7"
server_key_version: "encryptionkey@0"
I20260812 06:59:07.722319 6692 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260812 06:59:07.725085 6709 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 06:59:07.726538 6692 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 06:59:07.726711 6692 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786517687384214-19665-0/minicluster-data/master-0/data,/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786517687384214-19665-0/minicluster-data/master-0/wal
uuid: "8801dcfd547a41afb3b537a64694b310"
format_stamp: "Formatted at 2026-08-12 06:59:07 on dist-test-slave-65mx"
server_key: "8f716b56fbcad317ed0511fecb5eb87c"
server_key_iv: "5024ee8d91efca6be3557a53632fcad7"
server_key_version: "encryptionkey@0"
I20260812 06:59:07.726874 6692 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786517687384214-19665-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786517687384214-19665-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786517687384214-19665-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:59:07.768592 6692 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 06:59:07.772009 6692 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 06:59:07.772243 6692 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:59:07.779747 6692 rpc_server.cc:307] RPC server started. Bound to: 127.19.52.126:44425
I20260812 06:59:07.779822 6766 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.52.126:44425 every 8 connection(s)
I20260812 06:59:07.780843 6692 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786517687384214-19665-0/minicluster-data/master-0/data/info.pb
I20260812 06:59:07.783705 6767 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 06:59:07.787125 19665 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu as pid 6692
I20260812 06:59:07.787268 19665 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786517687384214-19665-0/minicluster-data/master-0/wal/instance
I20260812 06:59:07.787580 19665 external_mini_cluster.cc:1428] Setting key a55b417cd1e0f93dc72f3bd4e1749256
I20260812 06:59:07.790169 6767 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8801dcfd547a41afb3b537a64694b310: Bootstrap starting.
I20260812 06:59:07.792513 6767 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8801dcfd547a41afb3b537a64694b310: Neither blocks nor log segments found. Creating new log.
I20260812 06:59:07.793401 6767 log.cc:824] T 00000000000000000000000000000000 P 8801dcfd547a41afb3b537a64694b310: Log is configured to *not* fsync() on all Append() calls
I20260812 06:59:07.796073 6767 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8801dcfd547a41afb3b537a64694b310: No bootstrap required, opened a new log
Aug 12 06:59:07 dist-test-slave-65mx krb5kdc[6678](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517947, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.19.52.126@KRBTEST.COM
I20260812 06:59:07.800459 6767 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8801dcfd547a41afb3b537a64694b310 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8801dcfd547a41afb3b537a64694b310" member_type: VOTER last_known_addr { host: "127.19.52.126" port: 44425 } }
I20260812 06:59:07.800813 6767 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8801dcfd547a41afb3b537a64694b310 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 06:59:07.800904 6767 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8801dcfd547a41afb3b537a64694b310 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8801dcfd547a41afb3b537a64694b310, State: Initialized, Role: FOLLOWER
I20260812 06:59:07.801402 6767 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8801dcfd547a41afb3b537a64694b310 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8801dcfd547a41afb3b537a64694b310" member_type: VOTER last_known_addr { host: "127.19.52.126" port: 44425 } }
I20260812 06:59:07.801571 6767 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8801dcfd547a41afb3b537a64694b310 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 06:59:07.801649 6767 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8801dcfd547a41afb3b537a64694b310 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 06:59:07.801754 6767 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8801dcfd547a41afb3b537a64694b310 [term 0 FOLLOWER]: Advancing to term 1
I20260812 06:59:07.802764 6767 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8801dcfd547a41afb3b537a64694b310 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8801dcfd547a41afb3b537a64694b310" member_type: VOTER last_known_addr { host: "127.19.52.126" port: 44425 } }
I20260812 06:59:07.803123 6767 leader_election.cc:302] T 00000000000000000000000000000000 P 8801dcfd547a41afb3b537a64694b310 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8801dcfd547a41afb3b537a64694b310; no voters:
I20260812 06:59:07.803498 6767 leader_election.cc:288] T 00000000000000000000000000000000 P 8801dcfd547a41afb3b537a64694b310 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 06:59:07.803612 6772 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8801dcfd547a41afb3b537a64694b310 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 06:59:07.803859 6772 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8801dcfd547a41afb3b537a64694b310 [term 1 LEADER]: Becoming Leader. State: Replica: 8801dcfd547a41afb3b537a64694b310, State: Running, Role: LEADER
I20260812 06:59:07.804225 6772 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8801dcfd547a41afb3b537a64694b310 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8801dcfd547a41afb3b537a64694b310" member_type: VOTER last_known_addr { host: "127.19.52.126" port: 44425 } }
I20260812 06:59:07.804991 6767 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8801dcfd547a41afb3b537a64694b310 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 06:59:07.805053 6770 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:59:07.789275 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39328 (local address 127.19.52.126:44425)
0812 06:59:07.789772 (+ 497us) server_negotiation.cc:206] Beginning negotiation
0812 06:59:07.789842 (+ 70us) server_negotiation.cc:399] Waiting for connection header
0812 06:59:07.789908 (+ 66us) server_negotiation.cc:407] Connection header received
0812 06:59:07.790869 (+ 961us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:59:07.790891 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:59:07.791257 (+ 366us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:59:07.791634 (+ 377us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:59:07.792596 (+ 962us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:59:07.793905 (+ 1309us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:59:07.795388 (+ 1483us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:59:07.795913 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:59:07.796049 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:59:07.799173 (+ 3124us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:59:07.799179 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:59:07.799195 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:59:07.799228 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:59:07.801381 (+ 2153us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:59:07.802056 (+ 675us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:59:07.802061 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:59:07.802066 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:59:07.802201 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:59:07.802522 (+ 321us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:59:07.802526 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:59:07.802528 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:59:07.803063 (+ 535us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:59:07.803287 (+ 224us) server_negotiation.cc:1035] Waiting for connection context
0812 06:59:07.803597 (+ 310us) server_negotiation.cc:299] Negotiation successful
0812 06:59:07.804104 (+ 507us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":341,"thread_start_us":154,"threads_started":1}
I20260812 06:59:07.806079 6773 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8801dcfd547a41afb3b537a64694b310 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8801dcfd547a41afb3b537a64694b310" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8801dcfd547a41afb3b537a64694b310" member_type: VOTER last_known_addr { host: "127.19.52.126" port: 44425 } } }
I20260812 06:59:07.806222 6773 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8801dcfd547a41afb3b537a64694b310 [sys.catalog]: This master's current role is: LEADER
I20260812 06:59:07.806586 6774 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8801dcfd547a41afb3b537a64694b310 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8801dcfd547a41afb3b537a64694b310. Latest consensus state: current_term: 1 leader_uuid: "8801dcfd547a41afb3b537a64694b310" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8801dcfd547a41afb3b537a64694b310" member_type: VOTER last_known_addr { host: "127.19.52.126" port: 44425 } } }
I20260812 06:59:07.806679 6774 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8801dcfd547a41afb3b537a64694b310 [sys.catalog]: This master's current role is: LEADER
W20260812 06:59:07.810289 6788 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 8801dcfd547a41afb3b537a64694b310: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 06:59:07.810393 6788 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 06:59:07.810474 6781 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 06:59:07.811195 6781 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 06:59:07.816880 6781 catalog_manager.cc:1347] Generated new cluster ID: dc93c0a3512b4eeba5c4f32a9fddfb2c
I20260812 06:59:07.816965 6781 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 06:59:07.824726 6781 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 06:59:07.826005 6781 catalog_manager.cc:1504] Loading token signing keys...
I20260812 06:59:07.830657 6781 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8801dcfd547a41afb3b537a64694b310: Generated new TSK 0
I20260812 06:59:07.831434 6781 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.19.52.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.52.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.52.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786517687384214-19665-0/krb5kdc/kudu_127.19.52.65.keytab.
Entry for principal kudu/127.19.52.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786517687384214-19665-0/krb5kdc/kudu_127.19.52.65.keytab.
I20260812 06:59:07.871727 19665 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu
/tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786517687384214-19665-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786517687384214-19665-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786517687384214-19665-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786517687384214-19665-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.52.65:0
--local_ip_for_outbound_sockets=127.19.52.65
--webserver_interface=127.19.52.65
--webserver_port=0
--tserver_master_addrs=127.19.52.126:44425
--keytab_file=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786517687384214-19665-0/krb5kdc/kudu_127.19.52.65.keytab
--principal=kudu/127.19.52.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.52.84:46593
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786517687384214-19665-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786517687384214-19665-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786517687384214-19665-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 06:59:07.992959 6795 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:59:07.993294 6795 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:59:07.993388 6795 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 06:59:07.997618 6795 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 06:59:07.997715 6795 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 06:59:07.997805 6795 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.52.65
I20260812 06:59:08.003562 6795 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.52.84:46593
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786517687384214-19665-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786517687384214-19665-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786517687384214-19665-0/krb5kdc/kudu_127.19.52.65.keytab
--principal=kudu/127.19.52.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.52.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786517687384214-19665-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.19.52.65
--webserver_port=0
--tserver_master_addrs=127.19.52.126:44425
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6795
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.52.65
--log_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786517687384214-19665-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 06:15:54 UTC on ccycloud.quasar-rnfzce.root.comops.site
build id 4581989
I20260812 06:59:08.004848 6795 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:59:08.006067 6795 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 06:59:08.017475 6801 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:59:08.017453 6803 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:59:08.017432 6800 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:59:08.017612 6795 server_base.cc:1034] running on GCE node
I20260812 06:59:08.018131 6795 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:59:08.018882 6795 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:59:08.020074 6795 hybrid_clock.cc:648] HybridClock initialized: now 1786517948020051 us; error 34 us; skew 500 ppm
Aug 12 06:59:08 dist-test-slave-65mx krb5kdc[6678](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517948, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.52.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 06:59:08.023386 6795 init.cc:377] Logged in from keytab as kudu/127.19.52.65@KRBTEST.COM (short username kudu)
I20260812 06:59:08.024433 6795 webserver.cc:460] Webserver started at http://127.19.52.65:40603/ using document root <none> and password file <none>
I20260812 06:59:08.025084 6795 fs_manager.cc:359] Metadata directory not provided
I20260812 06:59:08.025166 6795 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:59:08.025404 6795 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 06:59:08.027693 6795 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786517687384214-19665-0/minicluster-data/ts-0/data/instance:
uuid: "996203e4c12d47d5bfd91244de82609c"
format_stamp: "Formatted at 2026-08-12 06:59:08 on dist-test-slave-65mx"
server_key: "af08254ea65629ead61863e3dfcb011d"
server_key_iv: "39dc2614c03cf3f52742b886dffc1b7b"
server_key_version: "encryptionkey@0"
I20260812 06:59:08.028332 6795 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786517687384214-19665-0/minicluster-data/ts-0/wal/instance:
uuid: "996203e4c12d47d5bfd91244de82609c"
format_stamp: "Formatted at 2026-08-12 06:59:08 on dist-test-slave-65mx"
server_key: "af08254ea65629ead61863e3dfcb011d"
server_key_iv: "39dc2614c03cf3f52742b886dffc1b7b"
server_key_version: "encryptionkey@0"
I20260812 06:59:08.032713 6795 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.002s
I20260812 06:59:08.035504 6813 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 06:59:08.036985 6795 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 06:59:08.037134 6795 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786517687384214-19665-0/minicluster-data/ts-0/data,/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786517687384214-19665-0/minicluster-data/ts-0/wal
uuid: "996203e4c12d47d5bfd91244de82609c"
format_stamp: "Formatted at 2026-08-12 06:59:08 on dist-test-slave-65mx"
server_key: "af08254ea65629ead61863e3dfcb011d"
server_key_iv: "39dc2614c03cf3f52742b886dffc1b7b"
server_key_version: "encryptionkey@0"
I20260812 06:59:08.037258 6795 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786517687384214-19665-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786517687384214-19665-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786517687384214-19665-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:59:08.068728 6795 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 06:59:08.072793 6795 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 06:59:08.072996 6795 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:59:08.073909 6795 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 06:59:08.074919 6795 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 06:59:08.074973 6795 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 06:59:08.075093 6795 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 06:59:08.075150 6795 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 06:59:08.086210 6795 rpc_server.cc:307] RPC server started. Bound to: 127.19.52.65:37543
I20260812 06:59:08.086243 6945 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.52.65:37543 every 8 connection(s)
I20260812 06:59:08.087276 6795 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786517687384214-19665-0/minicluster-data/ts-0/data/info.pb
I20260812 06:59:08.090541 19665 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu as pid 6795
I20260812 06:59:08.090724 19665 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786517687384214-19665-0/minicluster-data/ts-0/wal/instance
I20260812 06:59:08.091104 19665 external_mini_cluster.cc:1428] Setting key 85220f648c7c03c0fc3249c9f5e12b37
Aug 12 06:59:08 dist-test-slave-65mx krb5kdc[6678](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517948, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.52.65@KRBTEST.COM for kudu/127.19.52.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.19.52.66@KRBTEST.COM; defaulting to no policy
I20260812 06:59:08.105753 6770 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:59:08.089461 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.52.65:33837 (local address 127.19.52.126:44425)
0812 06:59:08.089642 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0812 06:59:08.089646 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 06:59:08.090704 (+ 1058us) server_negotiation.cc:407] Connection header received
0812 06:59:08.092792 (+ 2088us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:59:08.092795 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:59:08.092850 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:59:08.092988 (+ 138us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:59:08.094602 (+ 1614us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:59:08.095468 (+ 866us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:59:08.097541 (+ 2073us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:59:08.097830 (+ 289us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:59:08.097937 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:59:08.101279 (+ 3342us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:59:08.101283 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:59:08.101285 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:59:08.101316 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:59:08.103180 (+ 1864us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:59:08.103797 (+ 617us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:59:08.103800 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:59:08.103802 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:59:08.103894 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:59:08.104314 (+ 420us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:59:08.104317 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:59:08.104318 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:59:08.104636 (+ 318us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:59:08.104753 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0812 06:59:08.105385 (+ 632us) server_negotiation.cc:299] Negotiation successful
0812 06:59:08.105571 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.52.66@KRBTEST.COM" created.
I20260812 06:59:08.107081 6950 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:59:08.089886 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.126:44425 (local address 127.19.52.65:33837)
0812 06:59:08.090485 (+ 599us) negotiation.cc:107] Waiting for socket to connect
0812 06:59:08.090560 (+ 75us) client_negotiation.cc:174] Beginning negotiation
0812 06:59:08.092360 (+ 1800us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:59:08.093323 (+ 963us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:59:08.093331 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:59:08.093759 (+ 428us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:59:08.094315 (+ 556us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:59:08.094331 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:59:08.095673 (+ 1342us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:59:08.095678 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:59:08.097374 (+ 1696us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:59:08.097391 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:59:08.098030 (+ 639us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:59:08.098037 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:59:08.098291 (+ 254us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:59:08.099000 (+ 709us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 06:59:08.099028 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:59:08.101049 (+ 2021us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 06:59:08.103334 (+ 2285us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:59:08.103341 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:59:08.103359 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:59:08.103672 (+ 313us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:59:08.103996 (+ 324us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:59:08.104037 (+ 41us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:59:08.104039 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:59:08.104201 (+ 162us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:59:08.104753 (+ 552us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 06:59:08.104762 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 06:59:08.105055 (+ 293us) client_negotiation.cc:769] Sending connection context
0812 06:59:08.105325 (+ 270us) client_negotiation.cc:240] Negotiation successful
0812 06:59:08.105832 (+ 507us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":332,"thread_start_us":141,"threads_started":1}
I20260812 06:59:08.108637 6946 heartbeater.cc:347] Connected to a master server at 127.19.52.126:44425
I20260812 06:59:08.108918 6946 heartbeater.cc:464] Registering TS with master...
I20260812 06:59:08.109596 6946 heartbeater.cc:511] Master 127.19.52.126:44425 requested a full tablet report, sending...
I20260812 06:59:08.111348 6726 ts_manager.cc:194] Registered new tserver with Master: 996203e4c12d47d5bfd91244de82609c (127.19.52.65:37543)
I20260812 06:59:08.112625 6726 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.52.65@KRBTEST.COM'} at 127.19.52.65:33837
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.52.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786517687384214-19665-0/krb5kdc/kudu_127.19.52.66.keytab.
Entry for principal kudu/127.19.52.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786517687384214-19665-0/krb5kdc/kudu_127.19.52.66.keytab.
I20260812 06:59:08.128419 19665 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu
/tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786517687384214-19665-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786517687384214-19665-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786517687384214-19665-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786517687384214-19665-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.52.66:0
--local_ip_for_outbound_sockets=127.19.52.66
--webserver_interface=127.19.52.66
--webserver_port=0
--tserver_master_addrs=127.19.52.126:44425
--keytab_file=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786517687384214-19665-0/krb5kdc/kudu_127.19.52.66.keytab
--principal=kudu/127.19.52.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.52.84:46593
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786517687384214-19665-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786517687384214-19665-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786517687384214-19665-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 06:59:08.255717 6956 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:59:08.256148 6956 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:59:08.256208 6956 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 06:59:08.260921 6956 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 06:59:08.260999 6956 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 06:59:08.261061 6956 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.52.66
I20260812 06:59:08.266842 6956 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.52.84:46593
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786517687384214-19665-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786517687384214-19665-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786517687384214-19665-0/krb5kdc/kudu_127.19.52.66.keytab
--principal=kudu/127.19.52.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.52.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786517687384214-19665-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.19.52.66
--webserver_port=0
--tserver_master_addrs=127.19.52.126:44425
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6956
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.52.66
--log_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786517687384214-19665-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 06:15:54 UTC on ccycloud.quasar-rnfzce.root.comops.site
build id 4581989
I20260812 06:59:08.267943 6956 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:59:08.268963 6956 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 06:59:08.279386 6963 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:59:08.279386 6965 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:59:08.279358 6961 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:59:08.279656 6956 server_base.cc:1034] running on GCE node
I20260812 06:59:08.280304 6956 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:59:08.281203 6956 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:59:08.282478 6956 hybrid_clock.cc:648] HybridClock initialized: now 1786517948282418 us; error 85 us; skew 500 ppm
Aug 12 06:59:08 dist-test-slave-65mx krb5kdc[6678](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517948, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.52.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 06:59:08.285663 6956 init.cc:377] Logged in from keytab as kudu/127.19.52.66@KRBTEST.COM (short username kudu)
I20260812 06:59:08.286600 6956 webserver.cc:460] Webserver started at http://127.19.52.66:33577/ using document root <none> and password file <none>
I20260812 06:59:08.287209 6956 fs_manager.cc:359] Metadata directory not provided
I20260812 06:59:08.287272 6956 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:59:08.287499 6956 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 06:59:08.289750 6956 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786517687384214-19665-0/minicluster-data/ts-1/data/instance:
uuid: "4783465f20204a4ea07fc8f60a8b5cf0"
format_stamp: "Formatted at 2026-08-12 06:59:08 on dist-test-slave-65mx"
server_key: "4755b69a42dde8a88c8d91af2807f651"
server_key_iv: "5c3a343bda62d62289ba884d914b0cb2"
server_key_version: "encryptionkey@0"
I20260812 06:59:08.290373 6956 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786517687384214-19665-0/minicluster-data/ts-1/wal/instance:
uuid: "4783465f20204a4ea07fc8f60a8b5cf0"
format_stamp: "Formatted at 2026-08-12 06:59:08 on dist-test-slave-65mx"
server_key: "4755b69a42dde8a88c8d91af2807f651"
server_key_iv: "5c3a343bda62d62289ba884d914b0cb2"
server_key_version: "encryptionkey@0"
I20260812 06:59:08.294471 6956 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260812 06:59:08.297140 6972 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 06:59:08.298542 6956 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 06:59:08.298692 6956 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786517687384214-19665-0/minicluster-data/ts-1/data,/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786517687384214-19665-0/minicluster-data/ts-1/wal
uuid: "4783465f20204a4ea07fc8f60a8b5cf0"
format_stamp: "Formatted at 2026-08-12 06:59:08 on dist-test-slave-65mx"
server_key: "4755b69a42dde8a88c8d91af2807f651"
server_key_iv: "5c3a343bda62d62289ba884d914b0cb2"
server_key_version: "encryptionkey@0"
I20260812 06:59:08.298813 6956 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786517687384214-19665-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786517687384214-19665-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786517687384214-19665-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:59:08.309372 6956 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 06:59:08.312847 6956 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 06:59:08.313076 6956 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:59:08.313710 6956 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 06:59:08.314579 6956 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 06:59:08.314627 6956 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 06:59:08.314692 6956 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 06:59:08.314733 6956 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 06:59:08.324095 6956 rpc_server.cc:307] RPC server started. Bound to: 127.19.52.66:35099
I20260812 06:59:08.324115 7098 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.52.66:35099 every 8 connection(s)
I20260812 06:59:08.325170 6956 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786517687384214-19665-0/minicluster-data/ts-1/data/info.pb
I20260812 06:59:08.327522 19665 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu as pid 6956
I20260812 06:59:08.327668 19665 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786517687384214-19665-0/minicluster-data/ts-1/wal/instance
I20260812 06:59:08.328091 19665 external_mini_cluster.cc:1428] Setting key 6d7f9cb068f7c282a6a7bb85022ddc7b
Aug 12 06:59:08 dist-test-slave-65mx krb5kdc[6678](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517948, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.52.66@KRBTEST.COM for kudu/127.19.52.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.19.52.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.52.67@KRBTEST.COM" created.
I20260812 06:59:08.345139 6770 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:59:08.327196 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.52.66:55589 (local address 127.19.52.126:44425)
0812 06:59:08.327803 (+ 607us) server_negotiation.cc:206] Beginning negotiation
0812 06:59:08.327809 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 06:59:08.328290 (+ 481us) server_negotiation.cc:407] Connection header received
0812 06:59:08.330121 (+ 1831us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:59:08.330125 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:59:08.330170 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:59:08.330244 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:59:08.332484 (+ 2240us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:59:08.333117 (+ 633us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:59:08.334733 (+ 1616us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:59:08.335030 (+ 297us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:59:08.335133 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:59:08.339711 (+ 4578us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:59:08.339715 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:59:08.339717 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:59:08.339754 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:59:08.341683 (+ 1929us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:59:08.342696 (+ 1013us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:59:08.342699 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:59:08.342700 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:59:08.342798 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:59:08.343283 (+ 485us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:59:08.343286 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:59:08.343288 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:59:08.343814 (+ 526us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:59:08.343918 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0812 06:59:08.344711 (+ 793us) server_negotiation.cc:299] Negotiation successful
0812 06:59:08.344912 (+ 201us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":483}
I20260812 06:59:08.346850 7104 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:59:08.327493 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.126:44425 (local address 127.19.52.66:55589)
0812 06:59:08.328099 (+ 606us) negotiation.cc:107] Waiting for socket to connect
0812 06:59:08.328163 (+ 64us) client_negotiation.cc:174] Beginning negotiation
0812 06:59:08.329865 (+ 1702us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:59:08.330683 (+ 818us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:59:08.330694 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:59:08.331291 (+ 597us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:59:08.332250 (+ 959us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:59:08.332278 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:59:08.333268 (+ 990us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:59:08.333271 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:59:08.334548 (+ 1277us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:59:08.334563 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:59:08.335233 (+ 670us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:59:08.335241 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:59:08.335508 (+ 267us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:59:08.336423 (+ 915us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 06:59:08.336455 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:59:08.339361 (+ 2906us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 06:59:08.341866 (+ 2505us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:59:08.341872 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:59:08.341889 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:59:08.342549 (+ 660us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:59:08.342949 (+ 400us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:59:08.342951 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:59:08.342953 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:59:08.343144 (+ 191us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:59:08.343948 (+ 804us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 06:59:08.343955 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 06:59:08.344262 (+ 307us) client_negotiation.cc:769] Sending connection context
0812 06:59:08.344776 (+ 514us) client_negotiation.cc:240] Negotiation successful
0812 06:59:08.345411 (+ 635us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":365,"thread_start_us":161,"threads_started":1}
I20260812 06:59:08.348281 7102 heartbeater.cc:347] Connected to a master server at 127.19.52.126:44425
I20260812 06:59:08.348626 7102 heartbeater.cc:464] Registering TS with master...
I20260812 06:59:08.349215 7102 heartbeater.cc:511] Master 127.19.52.126:44425 requested a full tablet report, sending...
I20260812 06:59:08.350443 6726 ts_manager.cc:194] Registered new tserver with Master: 4783465f20204a4ea07fc8f60a8b5cf0 (127.19.52.66:35099)
I20260812 06:59:08.351173 6726 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.52.66@KRBTEST.COM'} at 127.19.52.66:55589
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.52.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786517687384214-19665-0/krb5kdc/kudu_127.19.52.67.keytab.
Entry for principal kudu/127.19.52.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786517687384214-19665-0/krb5kdc/kudu_127.19.52.67.keytab.
I20260812 06:59:08.364835 19665 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu
/tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786517687384214-19665-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786517687384214-19665-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786517687384214-19665-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786517687384214-19665-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.52.67:0
--local_ip_for_outbound_sockets=127.19.52.67
--webserver_interface=127.19.52.67
--webserver_port=0
--tserver_master_addrs=127.19.52.126:44425
--keytab_file=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786517687384214-19665-0/krb5kdc/kudu_127.19.52.67.keytab
--principal=kudu/127.19.52.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.52.84:46593
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786517687384214-19665-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786517687384214-19665-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786517687384214-19665-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 06:59:08.483453 7107 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:59:08.483796 7107 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:59:08.483884 7107 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 06:59:08.488152 7107 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 06:59:08.488263 7107 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 06:59:08.488356 7107 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.52.67
I20260812 06:59:08.494525 7107 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.52.84:46593
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786517687384214-19665-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786517687384214-19665-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786517687384214-19665-0/krb5kdc/kudu_127.19.52.67.keytab
--principal=kudu/127.19.52.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.52.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786517687384214-19665-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.19.52.67
--webserver_port=0
--tserver_master_addrs=127.19.52.126:44425
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7107
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.52.67
--log_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786517687384214-19665-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 06:15:54 UTC on ccycloud.quasar-rnfzce.root.comops.site
build id 4581989
I20260812 06:59:08.495749 7107 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:59:08.496857 7107 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 06:59:08.507040 7113 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:59:08.507050 7112 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:59:08.507340 7107 server_base.cc:1034] running on GCE node
W20260812 06:59:08.507051 7115 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:59:08.507891 7107 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:59:08.508657 7107 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:59:08.509822 7107 hybrid_clock.cc:648] HybridClock initialized: now 1786517948509792 us; error 42 us; skew 500 ppm
Aug 12 06:59:08 dist-test-slave-65mx krb5kdc[6678](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517948, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.52.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 06:59:08.512884 7107 init.cc:377] Logged in from keytab as kudu/127.19.52.67@KRBTEST.COM (short username kudu)
I20260812 06:59:08.513840 7107 webserver.cc:460] Webserver started at http://127.19.52.67:33841/ using document root <none> and password file <none>
I20260812 06:59:08.514439 7107 fs_manager.cc:359] Metadata directory not provided
I20260812 06:59:08.514545 7107 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:59:08.514789 7107 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 06:59:08.517048 7107 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786517687384214-19665-0/minicluster-data/ts-2/data/instance:
uuid: "46dbea4264a041668e92ef181c292a93"
format_stamp: "Formatted at 2026-08-12 06:59:08 on dist-test-slave-65mx"
server_key: "921798da1becd4ec2b13f8f8a53afb99"
server_key_iv: "ec2f8911782db196992474dc7e2d6f1e"
server_key_version: "encryptionkey@0"
I20260812 06:59:08.517640 7107 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786517687384214-19665-0/minicluster-data/ts-2/wal/instance:
uuid: "46dbea4264a041668e92ef181c292a93"
format_stamp: "Formatted at 2026-08-12 06:59:08 on dist-test-slave-65mx"
server_key: "921798da1becd4ec2b13f8f8a53afb99"
server_key_iv: "ec2f8911782db196992474dc7e2d6f1e"
server_key_version: "encryptionkey@0"
I20260812 06:59:08.521744 7107 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260812 06:59:08.524497 7122 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 06:59:08.525939 7107 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260812 06:59:08.526132 7107 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786517687384214-19665-0/minicluster-data/ts-2/data,/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786517687384214-19665-0/minicluster-data/ts-2/wal
uuid: "46dbea4264a041668e92ef181c292a93"
format_stamp: "Formatted at 2026-08-12 06:59:08 on dist-test-slave-65mx"
server_key: "921798da1becd4ec2b13f8f8a53afb99"
server_key_iv: "ec2f8911782db196992474dc7e2d6f1e"
server_key_version: "encryptionkey@0"
I20260812 06:59:08.526269 7107 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786517687384214-19665-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786517687384214-19665-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786517687384214-19665-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:59:08.548625 7107 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 06:59:08.552209 7107 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 06:59:08.552452 7107 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:59:08.553134 7107 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 06:59:08.554018 7107 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 06:59:08.554090 7107 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 06:59:08.554163 7107 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 06:59:08.554215 7107 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 06:59:08.563756 7107 rpc_server.cc:307] RPC server started. Bound to: 127.19.52.67:41523
I20260812 06:59:08.563779 7247 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.52.67:41523 every 8 connection(s)
I20260812 06:59:08.564742 7107 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786517687384214-19665-0/minicluster-data/ts-2/data/info.pb
I20260812 06:59:08.573836 19665 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu as pid 7107
I20260812 06:59:08.574033 19665 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786517687384214-19665-0/minicluster-data/ts-2/wal/instance
I20260812 06:59:08.574366 19665 external_mini_cluster.cc:1428] Setting key b83db2f031c6fec60139d2d28f10d1b3
Aug 12 06:59:08 dist-test-slave-65mx krb5kdc[6678](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517948, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.52.67@KRBTEST.COM for kudu/127.19.52.126@KRBTEST.COM
I20260812 06:59:08.580251 6770 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:59:08.566755 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.52.67:43145 (local address 127.19.52.126:44425)
0812 06:59:08.566957 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0812 06:59:08.566963 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 06:59:08.567885 (+ 922us) server_negotiation.cc:407] Connection header received
0812 06:59:08.569335 (+ 1450us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:59:08.569338 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:59:08.569392 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:59:08.569477 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:59:08.570756 (+ 1279us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:59:08.571348 (+ 592us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:59:08.572487 (+ 1139us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:59:08.572808 (+ 321us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:59:08.572898 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:59:08.575939 (+ 3041us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:59:08.576015 (+ 76us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:59:08.576017 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:59:08.576046 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:59:08.577766 (+ 1720us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:59:08.578288 (+ 522us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:59:08.578290 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:59:08.578291 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:59:08.578386 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:59:08.578779 (+ 393us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:59:08.578781 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:59:08.578782 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:59:08.578972 (+ 190us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:59:08.579061 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0812 06:59:08.579631 (+ 570us) server_negotiation.cc:299] Negotiation successful
0812 06:59:08.580116 (+ 485us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83}
I20260812 06:59:08.581177 7250 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:59:08.567199 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.126:44425 (local address 127.19.52.67:43145)
0812 06:59:08.567715 (+ 516us) negotiation.cc:107] Waiting for socket to connect
0812 06:59:08.567760 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0812 06:59:08.569104 (+ 1344us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:59:08.569714 (+ 610us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:59:08.569721 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:59:08.570105 (+ 384us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:59:08.570609 (+ 504us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:59:08.570624 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:59:08.571462 (+ 838us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:59:08.571464 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:59:08.572370 (+ 906us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:59:08.572379 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:59:08.572952 (+ 573us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:59:08.572955 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:59:08.573101 (+ 146us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:59:08.573851 (+ 750us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 06:59:08.573869 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:59:08.575780 (+ 1911us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 06:59:08.577880 (+ 2100us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:59:08.577885 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:59:08.577898 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:59:08.578193 (+ 295us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:59:08.578523 (+ 330us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:59:08.578525 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:59:08.578527 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:59:08.578685 (+ 158us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:59:08.579092 (+ 407us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 06:59:08.579097 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 06:59:08.579361 (+ 264us) client_negotiation.cc:769] Sending connection context
0812 06:59:08.579539 (+ 178us) client_negotiation.cc:240] Negotiation successful
0812 06:59:08.579817 (+ 278us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":308,"thread_start_us":134,"threads_started":1}
I20260812 06:59:08.582468 7248 heartbeater.cc:347] Connected to a master server at 127.19.52.126:44425
I20260812 06:59:08.582747 7248 heartbeater.cc:464] Registering TS with master...
I20260812 06:59:08.583302 7248 heartbeater.cc:511] Master 127.19.52.126:44425 requested a full tablet report, sending...
I20260812 06:59:08.584514 6726 ts_manager.cc:194] Registered new tserver with Master: 46dbea4264a041668e92ef181c292a93 (127.19.52.67:41523)
I20260812 06:59:08.585179 6726 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.52.67@KRBTEST.COM'} at 127.19.52.67:43145
I20260812 06:59:08.588821 19665 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 06:59:08.599133 6770 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:59:08.590887 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39340 (local address 127.19.52.126:44425)
0812 06:59:08.591017 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0812 06:59:08.591021 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 06:59:08.591259 (+ 238us) server_negotiation.cc:407] Connection header received
0812 06:59:08.591369 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:59:08.591371 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:59:08.591416 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:59:08.591485 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:59:08.592287 (+ 802us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:59:08.592953 (+ 666us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:59:08.593848 (+ 895us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:59:08.594078 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:59:08.594163 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:59:08.595670 (+ 1507us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:59:08.595673 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:59:08.595675 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:59:08.595700 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:59:08.597116 (+ 1416us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:59:08.597624 (+ 508us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:59:08.597629 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:59:08.597631 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:59:08.597843 (+ 212us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:59:08.598196 (+ 353us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:59:08.598199 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:59:08.598200 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:59:08.598461 (+ 261us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:59:08.598592 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0812 06:59:08.598847 (+ 255us) server_negotiation.cc:299] Negotiation successful
0812 06:59:08.598957 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260812 06:59:08.602557 6726 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39340:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 06:59:08.605847 6726 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 06:59:08.622414 7263 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:59:08.616337 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36740 (local address 127.19.52.66:35099)
0812 06:59:08.616866 (+ 529us) server_negotiation.cc:206] Beginning negotiation
0812 06:59:08.616886 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0812 06:59:08.616974 (+ 88us) server_negotiation.cc:407] Connection header received
0812 06:59:08.617120 (+ 146us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:59:08.617127 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:59:08.617366 (+ 239us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:59:08.617549 (+ 183us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 06:59:08.617904 (+ 355us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:59:08.618805 (+ 901us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:59:08.620454 (+ 1649us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:59:08.621567 (+ 1113us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:59:08.621718 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:59:08.621912 (+ 194us) server_negotiation.cc:1035] Waiting for connection context
0812 06:59:08.622117 (+ 205us) server_negotiation.cc:299] Negotiation successful
0812 06:59:08.622233 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":420,"thread_start_us":202,"threads_started":1}
I20260812 06:59:08.622416 7259 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:59:08.615649 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.66:35099 (local address 127.0.0.1:36740)
0812 06:59:08.616873 (+ 1224us) negotiation.cc:107] Waiting for socket to connect
0812 06:59:08.616885 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 06:59:08.617007 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:59:08.617549 (+ 542us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:59:08.617555 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:59:08.617577 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 06:59:08.617758 (+ 181us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:59:08.617769 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:59:08.618949 (+ 1180us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:59:08.618954 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:59:08.620312 (+ 1358us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:59:08.620323 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:59:08.621712 (+ 1389us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:59:08.621715 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:59:08.621887 (+ 172us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:59:08.621936 (+ 49us) client_negotiation.cc:769] Sending connection context
0812 06:59:08.622069 (+ 133us) client_negotiation.cc:240] Negotiation successful
0812 06:59:08.622224 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1166,"mutex_wait_us":48,"thread_start_us":123,"threads_started":1}
I20260812 06:59:08.622413 7260 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:59:08.615482 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.67:41523 (local address 127.0.0.1:40288)
0812 06:59:08.616472 (+ 990us) negotiation.cc:107] Waiting for socket to connect
0812 06:59:08.616524 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0812 06:59:08.616666 (+ 142us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:59:08.617705 (+ 1039us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:59:08.617711 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:59:08.617734 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 06:59:08.617868 (+ 134us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:59:08.617875 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:59:08.619513 (+ 1638us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:59:08.619516 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:59:08.620509 (+ 993us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:59:08.620520 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:59:08.621815 (+ 1295us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:59:08.621818 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:59:08.621947 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:59:08.621977 (+ 30us) client_negotiation.cc:769] Sending connection context
0812 06:59:08.622022 (+ 45us) client_negotiation.cc:240] Negotiation successful
0812 06:59:08.622237 (+ 215us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":791,"mutex_wait_us":83,"thread_start_us":101,"threads_started":1}
I20260812 06:59:08.623271 7262 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:59:08.615928 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.65:37543 (local address 127.0.0.1:34168)
0812 06:59:08.617275 (+ 1347us) negotiation.cc:107] Waiting for socket to connect
0812 06:59:08.617290 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 06:59:08.617396 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:59:08.617784 (+ 388us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:59:08.617788 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:59:08.617812 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 06:59:08.617926 (+ 114us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:59:08.617933 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:59:08.619513 (+ 1580us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:59:08.619516 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:59:08.621289 (+ 1773us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:59:08.621299 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:59:08.622656 (+ 1357us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:59:08.622660 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:59:08.622784 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:59:08.622808 (+ 24us) client_negotiation.cc:769] Sending connection context
0812 06:59:08.623019 (+ 211us) client_negotiation.cc:240] Negotiation successful
0812 06:59:08.623111 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1285,"mutex_wait_us":77,"thread_start_us":126,"threads_started":1}
I20260812 06:59:08.623746 7264 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:59:08.616602 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34168 (local address 127.19.52.65:37543)
0812 06:59:08.616919 (+ 317us) server_negotiation.cc:206] Beginning negotiation
0812 06:59:08.616941 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0812 06:59:08.617390 (+ 449us) server_negotiation.cc:407] Connection header received
0812 06:59:08.617507 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:59:08.617512 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:59:08.617673 (+ 161us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:59:08.617803 (+ 130us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 06:59:08.618088 (+ 285us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:59:08.619364 (+ 1276us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:59:08.621421 (+ 2057us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:59:08.622218 (+ 797us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:59:08.622321 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:59:08.622454 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0812 06:59:08.623449 (+ 995us) server_negotiation.cc:299] Negotiation successful
0812 06:59:08.623558 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":207,"thread_start_us":104,"threads_started":1}
I20260812 06:59:08.624338 7261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:59:08.615651 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40288 (local address 127.19.52.67:41523)
0812 06:59:08.617258 (+ 1607us) server_negotiation.cc:206] Beginning negotiation
0812 06:59:08.617266 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 06:59:08.617284 (+ 18us) server_negotiation.cc:407] Connection header received
0812 06:59:08.617341 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:59:08.617365 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:59:08.617543 (+ 178us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:59:08.618110 (+ 567us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 06:59:08.618285 (+ 175us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:59:08.619364 (+ 1079us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:59:08.620678 (+ 1314us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:59:08.621653 (+ 975us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:59:08.622940 (+ 1287us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:59:08.623105 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0812 06:59:08.624134 (+ 1029us) server_negotiation.cc:299] Negotiation successful
0812 06:59:08.624226 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1516,"thread_start_us":121,"threads_started":1}
I20260812 06:59:08.626263 6869 tablet_service.cc:1467] Processing CreateTablet for tablet 2748505f288046a29fc1499a1122954c (DEFAULT_TABLE table=test-table [id=b7d0b38accf3499cb4a0c96dd48cae96]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 06:59:08.626327 7022 tablet_service.cc:1467] Processing CreateTablet for tablet 2748505f288046a29fc1499a1122954c (DEFAULT_TABLE table=test-table [id=b7d0b38accf3499cb4a0c96dd48cae96]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 06:59:08.626928 7171 tablet_service.cc:1467] Processing CreateTablet for tablet 2748505f288046a29fc1499a1122954c (DEFAULT_TABLE table=test-table [id=b7d0b38accf3499cb4a0c96dd48cae96]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 06:59:08.627385 7022 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2748505f288046a29fc1499a1122954c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 06:59:08.627384 6869 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2748505f288046a29fc1499a1122954c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 06:59:08.627837 7171 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2748505f288046a29fc1499a1122954c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 06:59:08.634627 7265 tablet_bootstrap.cc:492] T 2748505f288046a29fc1499a1122954c P 996203e4c12d47d5bfd91244de82609c: Bootstrap starting.
I20260812 06:59:08.636036 7266 tablet_bootstrap.cc:492] T 2748505f288046a29fc1499a1122954c P 4783465f20204a4ea07fc8f60a8b5cf0: Bootstrap starting.
I20260812 06:59:08.636036 7267 tablet_bootstrap.cc:492] T 2748505f288046a29fc1499a1122954c P 46dbea4264a041668e92ef181c292a93: Bootstrap starting.
I20260812 06:59:08.637143 7265 tablet_bootstrap.cc:654] T 2748505f288046a29fc1499a1122954c P 996203e4c12d47d5bfd91244de82609c: Neither blocks nor log segments found. Creating new log.
I20260812 06:59:08.638072 7265 log.cc:824] T 2748505f288046a29fc1499a1122954c P 996203e4c12d47d5bfd91244de82609c: Log is configured to *not* fsync() on all Append() calls
I20260812 06:59:08.638320 7267 tablet_bootstrap.cc:654] T 2748505f288046a29fc1499a1122954c P 46dbea4264a041668e92ef181c292a93: Neither blocks nor log segments found. Creating new log.
I20260812 06:59:08.638976 7266 tablet_bootstrap.cc:654] T 2748505f288046a29fc1499a1122954c P 4783465f20204a4ea07fc8f60a8b5cf0: Neither blocks nor log segments found. Creating new log.
I20260812 06:59:08.639928 7266 log.cc:824] T 2748505f288046a29fc1499a1122954c P 4783465f20204a4ea07fc8f60a8b5cf0: Log is configured to *not* fsync() on all Append() calls
I20260812 06:59:08.639928 7267 log.cc:824] T 2748505f288046a29fc1499a1122954c P 46dbea4264a041668e92ef181c292a93: Log is configured to *not* fsync() on all Append() calls
I20260812 06:59:08.640323 7265 tablet_bootstrap.cc:492] T 2748505f288046a29fc1499a1122954c P 996203e4c12d47d5bfd91244de82609c: No bootstrap required, opened a new log
I20260812 06:59:08.640576 7265 ts_tablet_manager.cc:1397] T 2748505f288046a29fc1499a1122954c P 996203e4c12d47d5bfd91244de82609c: Time spent bootstrapping tablet: real 0.006s user 0.006s sys 0.000s
I20260812 06:59:08.642237 7267 tablet_bootstrap.cc:492] T 2748505f288046a29fc1499a1122954c P 46dbea4264a041668e92ef181c292a93: No bootstrap required, opened a new log
I20260812 06:59:08.642233 7266 tablet_bootstrap.cc:492] T 2748505f288046a29fc1499a1122954c P 4783465f20204a4ea07fc8f60a8b5cf0: No bootstrap required, opened a new log
I20260812 06:59:08.642454 7267 ts_tablet_manager.cc:1397] T 2748505f288046a29fc1499a1122954c P 46dbea4264a041668e92ef181c292a93: Time spent bootstrapping tablet: real 0.007s user 0.004s sys 0.000s
I20260812 06:59:08.642455 7266 ts_tablet_manager.cc:1397] T 2748505f288046a29fc1499a1122954c P 4783465f20204a4ea07fc8f60a8b5cf0: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260812 06:59:08.645510 7265 raft_consensus.cc:348] T 2748505f288046a29fc1499a1122954c P 996203e4c12d47d5bfd91244de82609c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46dbea4264a041668e92ef181c292a93" member_type: VOTER last_known_addr { host: "127.19.52.67" port: 41523 } } peers { permanent_uuid: "4783465f20204a4ea07fc8f60a8b5cf0" member_type: VOTER last_known_addr { host: "127.19.52.66" port: 35099 } } peers { permanent_uuid: "996203e4c12d47d5bfd91244de82609c" member_type: VOTER last_known_addr { host: "127.19.52.65" port: 37543 } }
I20260812 06:59:08.645510 7267 raft_consensus.cc:348] T 2748505f288046a29fc1499a1122954c P 46dbea4264a041668e92ef181c292a93 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46dbea4264a041668e92ef181c292a93" member_type: VOTER last_known_addr { host: "127.19.52.67" port: 41523 } } peers { permanent_uuid: "4783465f20204a4ea07fc8f60a8b5cf0" member_type: VOTER last_known_addr { host: "127.19.52.66" port: 35099 } } peers { permanent_uuid: "996203e4c12d47d5bfd91244de82609c" member_type: VOTER last_known_addr { host: "127.19.52.65" port: 37543 } }
I20260812 06:59:08.645838 7265 raft_consensus.cc:374] T 2748505f288046a29fc1499a1122954c P 996203e4c12d47d5bfd91244de82609c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 06:59:08.645838 7267 raft_consensus.cc:374] T 2748505f288046a29fc1499a1122954c P 46dbea4264a041668e92ef181c292a93 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 06:59:08.645900 7265 raft_consensus.cc:729] T 2748505f288046a29fc1499a1122954c P 996203e4c12d47d5bfd91244de82609c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 996203e4c12d47d5bfd91244de82609c, State: Initialized, Role: FOLLOWER
I20260812 06:59:08.645905 7267 raft_consensus.cc:729] T 2748505f288046a29fc1499a1122954c P 46dbea4264a041668e92ef181c292a93 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 46dbea4264a041668e92ef181c292a93, State: Initialized, Role: FOLLOWER
I20260812 06:59:08.646358 7267 consensus_queue.cc:260] T 2748505f288046a29fc1499a1122954c P 46dbea4264a041668e92ef181c292a93 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46dbea4264a041668e92ef181c292a93" member_type: VOTER last_known_addr { host: "127.19.52.67" port: 41523 } } peers { permanent_uuid: "4783465f20204a4ea07fc8f60a8b5cf0" member_type: VOTER last_known_addr { host: "127.19.52.66" port: 35099 } } peers { permanent_uuid: "996203e4c12d47d5bfd91244de82609c" member_type: VOTER last_known_addr { host: "127.19.52.65" port: 37543 } }
I20260812 06:59:08.646365 7265 consensus_queue.cc:260] T 2748505f288046a29fc1499a1122954c P 996203e4c12d47d5bfd91244de82609c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46dbea4264a041668e92ef181c292a93" member_type: VOTER last_known_addr { host: "127.19.52.67" port: 41523 } } peers { permanent_uuid: "4783465f20204a4ea07fc8f60a8b5cf0" member_type: VOTER last_known_addr { host: "127.19.52.66" port: 35099 } } peers { permanent_uuid: "996203e4c12d47d5bfd91244de82609c" member_type: VOTER last_known_addr { host: "127.19.52.65" port: 37543 } }
I20260812 06:59:08.646401 7266 raft_consensus.cc:348] T 2748505f288046a29fc1499a1122954c P 4783465f20204a4ea07fc8f60a8b5cf0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46dbea4264a041668e92ef181c292a93" member_type: VOTER last_known_addr { host: "127.19.52.67" port: 41523 } } peers { permanent_uuid: "4783465f20204a4ea07fc8f60a8b5cf0" member_type: VOTER last_known_addr { host: "127.19.52.66" port: 35099 } } peers { permanent_uuid: "996203e4c12d47d5bfd91244de82609c" member_type: VOTER last_known_addr { host: "127.19.52.65" port: 37543 } }
I20260812 06:59:08.646709 7266 raft_consensus.cc:374] T 2748505f288046a29fc1499a1122954c P 4783465f20204a4ea07fc8f60a8b5cf0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 06:59:08.646839 7266 raft_consensus.cc:729] T 2748505f288046a29fc1499a1122954c P 4783465f20204a4ea07fc8f60a8b5cf0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4783465f20204a4ea07fc8f60a8b5cf0, State: Initialized, Role: FOLLOWER
I20260812 06:59:08.647123 7248 heartbeater.cc:503] Master 127.19.52.126:44425 was elected leader, sending a full tablet report...
I20260812 06:59:08.647421 7267 ts_tablet_manager.cc:1428] T 2748505f288046a29fc1499a1122954c P 46dbea4264a041668e92ef181c292a93: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260812 06:59:08.647439 7265 ts_tablet_manager.cc:1428] T 2748505f288046a29fc1499a1122954c P 996203e4c12d47d5bfd91244de82609c: Time spent starting tablet: real 0.007s user 0.006s sys 0.000s
I20260812 06:59:08.647779 6946 heartbeater.cc:503] Master 127.19.52.126:44425 was elected leader, sending a full tablet report...
I20260812 06:59:08.648499 7266 consensus_queue.cc:260] T 2748505f288046a29fc1499a1122954c P 4783465f20204a4ea07fc8f60a8b5cf0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46dbea4264a041668e92ef181c292a93" member_type: VOTER last_known_addr { host: "127.19.52.67" port: 41523 } } peers { permanent_uuid: "4783465f20204a4ea07fc8f60a8b5cf0" member_type: VOTER last_known_addr { host: "127.19.52.66" port: 35099 } } peers { permanent_uuid: "996203e4c12d47d5bfd91244de82609c" member_type: VOTER last_known_addr { host: "127.19.52.65" port: 37543 } }
I20260812 06:59:08.649749 7266 ts_tablet_manager.cc:1428] T 2748505f288046a29fc1499a1122954c P 4783465f20204a4ea07fc8f60a8b5cf0: Time spent starting tablet: real 0.007s user 0.007s sys 0.000s
I20260812 06:59:08.650107 7102 heartbeater.cc:503] Master 127.19.52.126:44425 was elected leader, sending a full tablet report...
W20260812 06:59:08.815804 7249 tablet.cc:2369] T 2748505f288046a29fc1499a1122954c P 46dbea4264a041668e92ef181c292a93: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 06:59:08.826436 7103 tablet.cc:2369] T 2748505f288046a29fc1499a1122954c P 4783465f20204a4ea07fc8f60a8b5cf0: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 06:59:08.828356 7273 raft_consensus.cc:482] T 2748505f288046a29fc1499a1122954c P 4783465f20204a4ea07fc8f60a8b5cf0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 06:59:08.828594 7273 raft_consensus.cc:504] T 2748505f288046a29fc1499a1122954c P 4783465f20204a4ea07fc8f60a8b5cf0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46dbea4264a041668e92ef181c292a93" member_type: VOTER last_known_addr { host: "127.19.52.67" port: 41523 } } peers { permanent_uuid: "4783465f20204a4ea07fc8f60a8b5cf0" member_type: VOTER last_known_addr { host: "127.19.52.66" port: 35099 } } peers { permanent_uuid: "996203e4c12d47d5bfd91244de82609c" member_type: VOTER last_known_addr { host: "127.19.52.65" port: 37543 } }
I20260812 06:59:08.829823 7273 leader_election.cc:288] T 2748505f288046a29fc1499a1122954c P 4783465f20204a4ea07fc8f60a8b5cf0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 46dbea4264a041668e92ef181c292a93 (127.19.52.67:41523), 996203e4c12d47d5bfd91244de82609c (127.19.52.65:37543)
I20260812 06:59:08.834264 7264 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:59:08.830246 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.52.66:49397 (local address 127.19.52.65:37543)
0812 06:59:08.830391 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0812 06:59:08.830396 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 06:59:08.830412 (+ 16us) server_negotiation.cc:407] Connection header received
0812 06:59:08.830535 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:59:08.830539 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:59:08.830600 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:59:08.830677 (+ 77us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 06:59:08.831094 (+ 417us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:59:08.831783 (+ 689us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:59:08.833017 (+ 1234us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:59:08.833652 (+ 635us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:59:08.833753 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:59:08.833796 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0812 06:59:08.834092 (+ 296us) server_negotiation.cc:299] Negotiation successful
0812 06:59:08.834156 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260812 06:59:08.834319 7104 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:59:08.830122 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.65:37543 (local address 127.19.52.66:49397)
0812 06:59:08.830284 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0812 06:59:08.830303 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0812 06:59:08.830416 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:59:08.830746 (+ 330us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:59:08.830753 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:59:08.830812 (+ 59us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 06:59:08.830952 (+ 140us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:59:08.830961 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:59:08.831896 (+ 935us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:59:08.831900 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:59:08.832884 (+ 984us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:59:08.832894 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:59:08.833804 (+ 910us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:59:08.833809 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:59:08.833934 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:59:08.833962 (+ 28us) client_negotiation.cc:769] Sending connection context
0812 06:59:08.834071 (+ 109us) client_negotiation.cc:240] Negotiation successful
0812 06:59:08.834155 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":68,"spinlock_wait_cycles":7808}
I20260812 06:59:08.835063 7274 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:59:08.830122 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.67:41523 (local address 127.19.52.66:38555)
0812 06:59:08.830669 (+ 547us) negotiation.cc:107] Waiting for socket to connect
0812 06:59:08.830682 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 06:59:08.830815 (+ 133us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:59:08.831093 (+ 278us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:59:08.831096 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:59:08.831114 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 06:59:08.831236 (+ 122us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:59:08.831242 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:59:08.832253 (+ 1011us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:59:08.832256 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:59:08.833363 (+ 1107us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:59:08.833374 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:59:08.834553 (+ 1179us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:59:08.834559 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:59:08.834688 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:59:08.834708 (+ 20us) client_negotiation.cc:769] Sending connection context
0812 06:59:08.834806 (+ 98us) client_negotiation.cc:240] Negotiation successful
0812 06:59:08.834884 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":464,"mutex_wait_us":138,"spinlock_wait_cycles":44288,"thread_start_us":94,"threads_started":1}
I20260812 06:59:08.835039 7261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:59:08.830378 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.52.66:38555 (local address 127.19.52.67:41523)
0812 06:59:08.830684 (+ 306us) server_negotiation.cc:206] Beginning negotiation
0812 06:59:08.830690 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 06:59:08.830759 (+ 69us) server_negotiation.cc:407] Connection header received
0812 06:59:08.830918 (+ 159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:59:08.830923 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:59:08.830972 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:59:08.831054 (+ 82us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 06:59:08.831344 (+ 290us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:59:08.832101 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:59:08.833557 (+ 1456us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:59:08.834292 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:59:08.834426 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:59:08.834481 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0812 06:59:08.834838 (+ 357us) server_negotiation.cc:299] Negotiation successful
0812 06:59:08.834918 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":144}
I20260812 06:59:08.835069 6892 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2748505f288046a29fc1499a1122954c" candidate_uuid: "4783465f20204a4ea07fc8f60a8b5cf0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "996203e4c12d47d5bfd91244de82609c" is_pre_election: true
I20260812 06:59:08.835482 6892 raft_consensus.cc:2454] T 2748505f288046a29fc1499a1122954c P 996203e4c12d47d5bfd91244de82609c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4783465f20204a4ea07fc8f60a8b5cf0 in term 0.
I20260812 06:59:08.835741 7196 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2748505f288046a29fc1499a1122954c" candidate_uuid: "4783465f20204a4ea07fc8f60a8b5cf0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "46dbea4264a041668e92ef181c292a93" is_pre_election: true
I20260812 06:59:08.836078 7196 raft_consensus.cc:2454] T 2748505f288046a29fc1499a1122954c P 46dbea4264a041668e92ef181c292a93 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4783465f20204a4ea07fc8f60a8b5cf0 in term 0.
I20260812 06:59:08.836084 6975 leader_election.cc:302] T 2748505f288046a29fc1499a1122954c P 4783465f20204a4ea07fc8f60a8b5cf0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4783465f20204a4ea07fc8f60a8b5cf0, 996203e4c12d47d5bfd91244de82609c; no voters:
I20260812 06:59:08.836388 7273 raft_consensus.cc:2781] T 2748505f288046a29fc1499a1122954c P 4783465f20204a4ea07fc8f60a8b5cf0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 06:59:08.836496 7273 raft_consensus.cc:482] T 2748505f288046a29fc1499a1122954c P 4783465f20204a4ea07fc8f60a8b5cf0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 06:59:08.836603 7273 raft_consensus.cc:3037] T 2748505f288046a29fc1499a1122954c P 4783465f20204a4ea07fc8f60a8b5cf0 [term 0 FOLLOWER]: Advancing to term 1
I20260812 06:59:08.837932 7273 raft_consensus.cc:504] T 2748505f288046a29fc1499a1122954c P 4783465f20204a4ea07fc8f60a8b5cf0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46dbea4264a041668e92ef181c292a93" member_type: VOTER last_known_addr { host: "127.19.52.67" port: 41523 } } peers { permanent_uuid: "4783465f20204a4ea07fc8f60a8b5cf0" member_type: VOTER last_known_addr { host: "127.19.52.66" port: 35099 } } peers { permanent_uuid: "996203e4c12d47d5bfd91244de82609c" member_type: VOTER last_known_addr { host: "127.19.52.65" port: 37543 } }
I20260812 06:59:08.838348 7273 leader_election.cc:288] T 2748505f288046a29fc1499a1122954c P 4783465f20204a4ea07fc8f60a8b5cf0 [CANDIDATE]: Term 1 election: Requested vote from peers 46dbea4264a041668e92ef181c292a93 (127.19.52.67:41523), 996203e4c12d47d5bfd91244de82609c (127.19.52.65:37543)
I20260812 06:59:08.838779 7196 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2748505f288046a29fc1499a1122954c" candidate_uuid: "4783465f20204a4ea07fc8f60a8b5cf0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "46dbea4264a041668e92ef181c292a93"
I20260812 06:59:08.838867 6892 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2748505f288046a29fc1499a1122954c" candidate_uuid: "4783465f20204a4ea07fc8f60a8b5cf0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "996203e4c12d47d5bfd91244de82609c"
I20260812 06:59:08.838951 7196 raft_consensus.cc:3037] T 2748505f288046a29fc1499a1122954c P 46dbea4264a041668e92ef181c292a93 [term 0 FOLLOWER]: Advancing to term 1
W20260812 06:59:08.839057 6947 tablet.cc:2369] T 2748505f288046a29fc1499a1122954c P 996203e4c12d47d5bfd91244de82609c: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 06:59:08.839128 6892 raft_consensus.cc:3037] T 2748505f288046a29fc1499a1122954c P 996203e4c12d47d5bfd91244de82609c [term 0 FOLLOWER]: Advancing to term 1
I20260812 06:59:08.840197 7196 raft_consensus.cc:2454] T 2748505f288046a29fc1499a1122954c P 46dbea4264a041668e92ef181c292a93 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4783465f20204a4ea07fc8f60a8b5cf0 in term 1.
I20260812 06:59:08.840212 6892 raft_consensus.cc:2454] T 2748505f288046a29fc1499a1122954c P 996203e4c12d47d5bfd91244de82609c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4783465f20204a4ea07fc8f60a8b5cf0 in term 1.
I20260812 06:59:08.840585 6974 leader_election.cc:302] T 2748505f288046a29fc1499a1122954c P 4783465f20204a4ea07fc8f60a8b5cf0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46dbea4264a041668e92ef181c292a93, 4783465f20204a4ea07fc8f60a8b5cf0; no voters:
I20260812 06:59:08.840812 7273 raft_consensus.cc:2781] T 2748505f288046a29fc1499a1122954c P 4783465f20204a4ea07fc8f60a8b5cf0 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 06:59:08.841046 7273 raft_consensus.cc:686] T 2748505f288046a29fc1499a1122954c P 4783465f20204a4ea07fc8f60a8b5cf0 [term 1 LEADER]: Becoming Leader. State: Replica: 4783465f20204a4ea07fc8f60a8b5cf0, State: Running, Role: LEADER
I20260812 06:59:08.841382 7273 consensus_queue.cc:237] T 2748505f288046a29fc1499a1122954c P 4783465f20204a4ea07fc8f60a8b5cf0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46dbea4264a041668e92ef181c292a93" member_type: VOTER last_known_addr { host: "127.19.52.67" port: 41523 } } peers { permanent_uuid: "4783465f20204a4ea07fc8f60a8b5cf0" member_type: VOTER last_known_addr { host: "127.19.52.66" port: 35099 } } peers { permanent_uuid: "996203e4c12d47d5bfd91244de82609c" member_type: VOTER last_known_addr { host: "127.19.52.65" port: 37543 } }
I20260812 06:59:08.844494 6727 catalog_manager.cc:5495] T 2748505f288046a29fc1499a1122954c P 4783465f20204a4ea07fc8f60a8b5cf0 reported cstate change: term changed from 0 to 1, leader changed from <none> to 4783465f20204a4ea07fc8f60a8b5cf0 (127.19.52.66). New cstate: current_term: 1 leader_uuid: "4783465f20204a4ea07fc8f60a8b5cf0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46dbea4264a041668e92ef181c292a93" member_type: VOTER last_known_addr { host: "127.19.52.67" port: 41523 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4783465f20204a4ea07fc8f60a8b5cf0" member_type: VOTER last_known_addr { host: "127.19.52.66" port: 35099 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "996203e4c12d47d5bfd91244de82609c" member_type: VOTER last_known_addr { host: "127.19.52.65" port: 37543 } health_report { overall_health: UNKNOWN } } }
I20260812 06:59:08.859668 7263 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:59:08.855847 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36756 (local address 127.19.52.66:35099)
0812 06:59:08.856033 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0812 06:59:08.856038 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 06:59:08.856053 (+ 15us) server_negotiation.cc:407] Connection header received
0812 06:59:08.856216 (+ 163us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:59:08.856218 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:59:08.856278 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:59:08.856395 (+ 117us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 06:59:08.856719 (+ 324us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:59:08.857264 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:59:08.858153 (+ 889us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:59:08.858379 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:59:08.858495 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:59:08.858808 (+ 313us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 06:59:08.859249 (+ 441us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 06:59:08.859388 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0812 06:59:08.859450 (+ 62us) server_negotiation.cc:299] Negotiation successful
0812 06:59:08.859513 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83}
I20260812 06:59:08.865711 7196 raft_consensus.cc:1261] T 2748505f288046a29fc1499a1122954c P 46dbea4264a041668e92ef181c292a93 [term 1 FOLLOWER]: Refusing update from remote peer 4783465f20204a4ea07fc8f60a8b5cf0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 06:59:08.865711 6892 raft_consensus.cc:1261] T 2748505f288046a29fc1499a1122954c P 996203e4c12d47d5bfd91244de82609c [term 1 FOLLOWER]: Refusing update from remote peer 4783465f20204a4ea07fc8f60a8b5cf0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 06:59:08.866431 7273 consensus_queue.cc:1035] T 2748505f288046a29fc1499a1122954c P 4783465f20204a4ea07fc8f60a8b5cf0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "46dbea4264a041668e92ef181c292a93" member_type: VOTER last_known_addr { host: "127.19.52.67" port: 41523 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 06:59:08.866667 7275 consensus_queue.cc:1035] T 2748505f288046a29fc1499a1122954c P 4783465f20204a4ea07fc8f60a8b5cf0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "996203e4c12d47d5bfd91244de82609c" member_type: VOTER last_known_addr { host: "127.19.52.65" port: 37543 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 06:59:08.874804 7280 mvcc.cc:205] Tried to move back new op lower bound from 7317577518546214912 to 7317577518457135104. Current Snapshot: MvccSnapshot[applied={T|T < 7317577518546214912}]
I20260812 06:59:08.875020 7279 mvcc.cc:205] Tried to move back new op lower bound from 7317577518546214912 to 7317577518457135104. Current Snapshot: MvccSnapshot[applied={T|T < 7317577518546214912}]
I20260812 06:59:08.885742 6726 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39340:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 06:59:08.885958 6726 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39340:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 06:59:08.888873 6726 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 8801dcfd547a41afb3b537a64694b310: Sending DeleteTablet for 3 replicas of tablet 2748505f288046a29fc1499a1122954c
I20260812 06:59:08.889793 7022 tablet_service.cc:1514] Processing DeleteTablet for tablet 2748505f288046a29fc1499a1122954c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 06:59:08 UTC) from {username='kudu', principal='kudu/127.19.52.126@KRBTEST.COM'} at 127.0.0.1:36740
I20260812 06:59:08.889940 6869 tablet_service.cc:1514] Processing DeleteTablet for tablet 2748505f288046a29fc1499a1122954c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 06:59:08 UTC) from {username='kudu', principal='kudu/127.19.52.126@KRBTEST.COM'} at 127.0.0.1:34168
I20260812 06:59:08.890470 7289 tablet_replica.cc:330] T 2748505f288046a29fc1499a1122954c P 996203e4c12d47d5bfd91244de82609c: stopping tablet replica
I20260812 06:59:08.890436 7171 tablet_service.cc:1514] Processing DeleteTablet for tablet 2748505f288046a29fc1499a1122954c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 06:59:08 UTC) from {username='kudu', principal='kudu/127.19.52.126@KRBTEST.COM'} at 127.0.0.1:40288
I20260812 06:59:08.890597 19665 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu with pid 6795
I20260812 06:59:08.895599 7288 tablet_replica.cc:330] T 2748505f288046a29fc1499a1122954c P 4783465f20204a4ea07fc8f60a8b5cf0: stopping tablet replica
I20260812 06:59:08.896046 7288 raft_consensus.cc:2229] T 2748505f288046a29fc1499a1122954c P 4783465f20204a4ea07fc8f60a8b5cf0 [term 1 LEADER]: Raft consensus shutting down.
I20260812 06:59:08.896648 7288 raft_consensus.cc:2258] T 2748505f288046a29fc1499a1122954c P 4783465f20204a4ea07fc8f60a8b5cf0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 06:59:08.896797 7290 tablet_replica.cc:330] T 2748505f288046a29fc1499a1122954c P 46dbea4264a041668e92ef181c292a93: stopping tablet replica
I20260812 06:59:08.897238 7290 raft_consensus.cc:2229] T 2748505f288046a29fc1499a1122954c P 46dbea4264a041668e92ef181c292a93 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 06:59:08.897588 7290 raft_consensus.cc:2258] T 2748505f288046a29fc1499a1122954c P 46dbea4264a041668e92ef181c292a93 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 06:59:08.899551 7290 ts_tablet_manager.cc:1905] T 2748505f288046a29fc1499a1122954c P 46dbea4264a041668e92ef181c292a93: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 06:59:08.899654 7288 ts_tablet_manager.cc:1905] T 2748505f288046a29fc1499a1122954c P 4783465f20204a4ea07fc8f60a8b5cf0: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260812 06:59:08.902354 6713 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.19.52.65:37543 (error 108)
I20260812 06:59:08.903107 7290 ts_tablet_manager.cc:1918] T 2748505f288046a29fc1499a1122954c P 46dbea4264a041668e92ef181c292a93: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 06:59:08.903086 7288 ts_tablet_manager.cc:1918] T 2748505f288046a29fc1499a1122954c P 4783465f20204a4ea07fc8f60a8b5cf0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 06:59:08.903277 7288 log.cc:1196] T 2748505f288046a29fc1499a1122954c P 4783465f20204a4ea07fc8f60a8b5cf0: Deleting WAL directory at /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786517687384214-19665-0/minicluster-data/ts-1/wal/wals/2748505f288046a29fc1499a1122954c
I20260812 06:59:08.903362 7290 log.cc:1196] T 2748505f288046a29fc1499a1122954c P 46dbea4264a041668e92ef181c292a93: Deleting WAL directory at /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786517687384214-19665-0/minicluster-data/ts-2/wal/wals/2748505f288046a29fc1499a1122954c
I20260812 06:59:08.903509 19665 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu with pid 6956
I20260812 06:59:08.903895 7288 ts_tablet_manager.cc:1939] T 2748505f288046a29fc1499a1122954c P 4783465f20204a4ea07fc8f60a8b5cf0: Deleting consensus metadata
I20260812 06:59:08.905699 7259 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:59:08.905305 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.65:37543 (local address 127.0.0.1:34180)
0812 06:59:08.905521 (+ 216us) negotiation.cc:107] Waiting for socket to connect
0812 06:59:08.905596 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.52.65:37543: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":119}
W20260812 06:59:08.905920 6713 catalog_manager.cc:4584] TS 996203e4c12d47d5bfd91244de82609c (127.19.52.65:37543): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 2748505f288046a29fc1499a1122954c: Network error: Client connection negotiation failed: client connection to 127.19.52.65:37543: connect: Connection refused (error 111)
I20260812 06:59:08.906332 7290 ts_tablet_manager.cc:1939] T 2748505f288046a29fc1499a1122954c P 46dbea4264a041668e92ef181c292a93: Deleting consensus metadata
I20260812 06:59:08.907763 6711 catalog_manager.cc:4841] TS 46dbea4264a041668e92ef181c292a93 (127.19.52.67:41523): tablet 2748505f288046a29fc1499a1122954c (table test-table [id=b7d0b38accf3499cb4a0c96dd48cae96]) successfully deleted
I20260812 06:59:08.913014 19665 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu with pid 7107
I20260812 06:59:08.913666 7259 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:59:08.913264 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.66:35099 (local address 127.0.0.1:36760)
0812 06:59:08.913480 (+ 216us) negotiation.cc:107] Waiting for socket to connect
0812 06:59:08.913557 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.52.66:35099: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":117}
I20260812 06:59:08.921471 19665 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu with pid 6692
2026-08-12T06:59:08Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/0 (2659 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/1
2026-08-12T06:59:08Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T06:59:08Z Disabled control of system clock
I20260812 06:59:08.959249 19665 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu
/tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786517687384214-19665-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786517687384214-19665-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786517687384214-19665-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786517687384214-19665-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.52.126:42821
--webserver_interface=127.19.52.126
--webserver_port=0
--builtin_ntp_servers=127.19.52.84:42247
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.19.52.126:42821
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260812 06:59:09.079422 7295 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:59:09.079746 7295 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:59:09.079785 7295 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 06:59:09.083973 7295 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 06:59:09.084061 7295 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 06:59:09.084079 7295 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 06:59:09.084095 7295 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 06:59:09.084111 7295 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 06:59:09.089159 7295 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.52.84:42247
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786517687384214-19665-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786517687384214-19665-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.19.52.126:42821
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.52.126:42821
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786517687384214-19665-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.19.52.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7295
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786517687384214-19665-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 06:15:54 UTC on ccycloud.quasar-rnfzce.root.comops.site
build id 4581989
I20260812 06:59:09.090317 7295 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:59:09.091758 7295 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 06:59:09.100695 7302 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:59:09.100733 7305 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:59:09.100646 7303 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:59:09.100924 7295 server_base.cc:1034] running on GCE node
I20260812 06:59:09.101688 7295 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:59:09.102900 7295 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:59:09.104116 7295 hybrid_clock.cc:648] HybridClock initialized: now 1786517949104099 us; error 36 us; skew 500 ppm
I20260812 06:59:09.106403 7295 webserver.cc:460] Webserver started at http://127.19.52.126:41305/ using document root <none> and password file <none>
I20260812 06:59:09.107030 7295 fs_manager.cc:359] Metadata directory not provided
I20260812 06:59:09.107131 7295 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:59:09.107373 7295 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 06:59:09.109607 7295 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786517687384214-19665-0/minicluster-data/master-0/data/instance:
uuid: "7f03d44499c24625ab8abadf93b87969"
format_stamp: "Formatted at 2026-08-12 06:59:09 on dist-test-slave-65mx"
server_key: "0b69a640c4653075f09dae6f41e5ce38"
server_key_iv: "ba857828b77a7304c32a0512772710c5"
server_key_version: "encryptionkey@0"
I20260812 06:59:09.110214 7295 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786517687384214-19665-0/minicluster-data/master-0/wal/instance:
uuid: "7f03d44499c24625ab8abadf93b87969"
format_stamp: "Formatted at 2026-08-12 06:59:09 on dist-test-slave-65mx"
server_key: "0b69a640c4653075f09dae6f41e5ce38"
server_key_iv: "ba857828b77a7304c32a0512772710c5"
server_key_version: "encryptionkey@0"
I20260812 06:59:09.114466 7295 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260812 06:59:09.117060 7312 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 06:59:09.118299 7295 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260812 06:59:09.118444 7295 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786517687384214-19665-0/minicluster-data/master-0/data,/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786517687384214-19665-0/minicluster-data/master-0/wal
uuid: "7f03d44499c24625ab8abadf93b87969"
format_stamp: "Formatted at 2026-08-12 06:59:09 on dist-test-slave-65mx"
server_key: "0b69a640c4653075f09dae6f41e5ce38"
server_key_iv: "ba857828b77a7304c32a0512772710c5"
server_key_version: "encryptionkey@0"
I20260812 06:59:09.118566 7295 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786517687384214-19665-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786517687384214-19665-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786517687384214-19665-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:59:09.151273 7295 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 06:59:09.152093 7295 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 06:59:09.152330 7295 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:59:09.159808 7295 rpc_server.cc:307] RPC server started. Bound to: 127.19.52.126:42821
I20260812 06:59:09.159819 7371 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.52.126:42821 every 8 connection(s)
I20260812 06:59:09.160856 7295 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786517687384214-19665-0/minicluster-data/master-0/data/info.pb
I20260812 06:59:09.164129 7373 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 06:59:09.167274 19665 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu as pid 7295
I20260812 06:59:09.167392 19665 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786517687384214-19665-0/minicluster-data/master-0/wal/instance
I20260812 06:59:09.167796 19665 external_mini_cluster.cc:1428] Setting key 21438c6aee4f1a5fdab784456bcfe412
I20260812 06:59:09.170521 7373 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7f03d44499c24625ab8abadf93b87969: Bootstrap starting.
I20260812 06:59:09.172876 7373 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7f03d44499c24625ab8abadf93b87969: Neither blocks nor log segments found. Creating new log.
I20260812 06:59:09.173651 7373 log.cc:824] T 00000000000000000000000000000000 P 7f03d44499c24625ab8abadf93b87969: Log is configured to *not* fsync() on all Append() calls
I20260812 06:59:09.175624 7373 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7f03d44499c24625ab8abadf93b87969: No bootstrap required, opened a new log
I20260812 06:59:09.177568 7376 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:59:09.169375 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33456 (local address 127.19.52.126:42821)
0812 06:59:09.170085 (+ 710us) server_negotiation.cc:206] Beginning negotiation
0812 06:59:09.170102 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0812 06:59:09.170139 (+ 37us) server_negotiation.cc:407] Connection header received
0812 06:59:09.170898 (+ 759us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:59:09.170917 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:59:09.171310 (+ 393us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:59:09.171653 (+ 343us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:59:09.172093 (+ 440us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:59:09.173194 (+ 1101us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:59:09.174124 (+ 930us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:59:09.174605 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:59:09.174775 (+ 170us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:59:09.175668 (+ 893us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:59:09.175674 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:59:09.175686 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 06:59:09.176180 (+ 494us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:59:09.176240 (+ 60us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 06:59:09.176262 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:59:09.176376 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0812 06:59:09.176497 (+ 121us) server_negotiation.cc:299] Negotiation successful
0812 06:59:09.176801 (+ 304us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":329,"thread_start_us":153,"threads_started":1}
I20260812 06:59:09.179800 7373 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7f03d44499c24625ab8abadf93b87969 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7f03d44499c24625ab8abadf93b87969" member_type: VOTER last_known_addr { host: "127.19.52.126" port: 42821 } }
I20260812 06:59:09.180130 7373 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7f03d44499c24625ab8abadf93b87969 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 06:59:09.180213 7373 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7f03d44499c24625ab8abadf93b87969 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7f03d44499c24625ab8abadf93b87969, State: Initialized, Role: FOLLOWER
I20260812 06:59:09.180697 7373 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7f03d44499c24625ab8abadf93b87969 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7f03d44499c24625ab8abadf93b87969" member_type: VOTER last_known_addr { host: "127.19.52.126" port: 42821 } }
I20260812 06:59:09.180855 7373 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7f03d44499c24625ab8abadf93b87969 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 06:59:09.180943 7373 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7f03d44499c24625ab8abadf93b87969 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 06:59:09.181064 7373 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7f03d44499c24625ab8abadf93b87969 [term 0 FOLLOWER]: Advancing to term 1
I20260812 06:59:09.182070 7373 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7f03d44499c24625ab8abadf93b87969 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7f03d44499c24625ab8abadf93b87969" member_type: VOTER last_known_addr { host: "127.19.52.126" port: 42821 } }
I20260812 06:59:09.182431 7373 leader_election.cc:302] T 00000000000000000000000000000000 P 7f03d44499c24625ab8abadf93b87969 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7f03d44499c24625ab8abadf93b87969; no voters:
I20260812 06:59:09.182771 7373 leader_election.cc:288] T 00000000000000000000000000000000 P 7f03d44499c24625ab8abadf93b87969 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 06:59:09.182972 7379 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7f03d44499c24625ab8abadf93b87969 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 06:59:09.183342 7379 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7f03d44499c24625ab8abadf93b87969 [term 1 LEADER]: Becoming Leader. State: Replica: 7f03d44499c24625ab8abadf93b87969, State: Running, Role: LEADER
I20260812 06:59:09.183650 7379 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7f03d44499c24625ab8abadf93b87969 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7f03d44499c24625ab8abadf93b87969" member_type: VOTER last_known_addr { host: "127.19.52.126" port: 42821 } }
I20260812 06:59:09.184176 7373 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7f03d44499c24625ab8abadf93b87969 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 06:59:09.185292 7381 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7f03d44499c24625ab8abadf93b87969 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7f03d44499c24625ab8abadf93b87969. Latest consensus state: current_term: 1 leader_uuid: "7f03d44499c24625ab8abadf93b87969" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7f03d44499c24625ab8abadf93b87969" member_type: VOTER last_known_addr { host: "127.19.52.126" port: 42821 } } }
I20260812 06:59:09.185421 7381 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7f03d44499c24625ab8abadf93b87969 [sys.catalog]: This master's current role is: LEADER
I20260812 06:59:09.185573 7380 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7f03d44499c24625ab8abadf93b87969 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7f03d44499c24625ab8abadf93b87969" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7f03d44499c24625ab8abadf93b87969" member_type: VOTER last_known_addr { host: "127.19.52.126" port: 42821 } } }
I20260812 06:59:09.185843 7380 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7f03d44499c24625ab8abadf93b87969 [sys.catalog]: This master's current role is: LEADER
I20260812 06:59:09.185878 7392 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 06:59:09.189203 7392 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 06:59:09.195164 7392 catalog_manager.cc:1347] Generated new cluster ID: 9ac3bca33e004b03bdc8689dd2e70c49
I20260812 06:59:09.195254 7392 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 06:59:09.207878 7392 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 06:59:09.209009 7392 catalog_manager.cc:1504] Loading token signing keys...
I20260812 06:59:09.218147 7392 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7f03d44499c24625ab8abadf93b87969: Generated new TSK 0
I20260812 06:59:09.218992 7392 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 06:59:09.222579 19665 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu
/tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786517687384214-19665-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786517687384214-19665-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786517687384214-19665-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786517687384214-19665-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.52.65:0
--local_ip_for_outbound_sockets=127.19.52.65
--webserver_interface=127.19.52.65
--webserver_port=0
--tserver_master_addrs=127.19.52.126:42821
--builtin_ntp_servers=127.19.52.84:42247
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260812 06:59:09.345687 7402 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:59:09.346009 7402 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:59:09.346057 7402 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 06:59:09.350837 7402 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 06:59:09.350926 7402 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 06:59:09.350989 7402 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.52.65
I20260812 06:59:09.356333 7402 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.52.84:42247
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786517687384214-19665-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786517687384214-19665-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.52.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786517687384214-19665-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.19.52.65
--webserver_port=0
--tserver_master_addrs=127.19.52.126:42821
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7402
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.52.65
--log_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786517687384214-19665-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 06:15:54 UTC on ccycloud.quasar-rnfzce.root.comops.site
build id 4581989
I20260812 06:59:09.357653 7402 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:59:09.359130 7402 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 06:59:09.369915 7411 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:59:09.369915 7409 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:59:09.369974 7408 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:59:09.370348 7402 server_base.cc:1034] running on GCE node
I20260812 06:59:09.370795 7402 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:59:09.371526 7402 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:59:09.372673 7402 hybrid_clock.cc:648] HybridClock initialized: now 1786517949372659 us; error 31 us; skew 500 ppm
I20260812 06:59:09.374868 7402 webserver.cc:460] Webserver started at http://127.19.52.65:38485/ using document root <none> and password file <none>
I20260812 06:59:09.375483 7402 fs_manager.cc:359] Metadata directory not provided
I20260812 06:59:09.375558 7402 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:59:09.375792 7402 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 06:59:09.378082 7402 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786517687384214-19665-0/minicluster-data/ts-0/data/instance:
uuid: "7e187814a4a7477e8c7c6385cd56a295"
format_stamp: "Formatted at 2026-08-12 06:59:09 on dist-test-slave-65mx"
server_key: "1f02142222398fc1e205a49f8d46f8b4"
server_key_iv: "7ac37df71a2f36a1ba92da67865f74f1"
server_key_version: "encryptionkey@0"
I20260812 06:59:09.378680 7402 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786517687384214-19665-0/minicluster-data/ts-0/wal/instance:
uuid: "7e187814a4a7477e8c7c6385cd56a295"
format_stamp: "Formatted at 2026-08-12 06:59:09 on dist-test-slave-65mx"
server_key: "1f02142222398fc1e205a49f8d46f8b4"
server_key_iv: "7ac37df71a2f36a1ba92da67865f74f1"
server_key_version: "encryptionkey@0"
I20260812 06:59:09.382997 7402 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260812 06:59:09.386041 7417 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 06:59:09.387627 7402 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 06:59:09.387807 7402 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786517687384214-19665-0/minicluster-data/ts-0/data,/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786517687384214-19665-0/minicluster-data/ts-0/wal
uuid: "7e187814a4a7477e8c7c6385cd56a295"
format_stamp: "Formatted at 2026-08-12 06:59:09 on dist-test-slave-65mx"
server_key: "1f02142222398fc1e205a49f8d46f8b4"
server_key_iv: "7ac37df71a2f36a1ba92da67865f74f1"
server_key_version: "encryptionkey@0"
I20260812 06:59:09.387989 7402 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786517687384214-19665-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786517687384214-19665-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786517687384214-19665-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:59:09.400738 7402 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 06:59:09.401544 7402 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 06:59:09.401767 7402 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:59:09.402410 7402 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 06:59:09.403417 7402 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 06:59:09.403494 7402 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 06:59:09.403574 7402 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 06:59:09.403620 7402 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 06:59:09.416682 7402 rpc_server.cc:307] RPC server started. Bound to: 127.19.52.65:45755
I20260812 06:59:09.416972 7546 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.52.65:45755 every 8 connection(s)
I20260812 06:59:09.418013 7402 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786517687384214-19665-0/minicluster-data/ts-0/data/info.pb
I20260812 06:59:09.421396 19665 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu as pid 7402
I20260812 06:59:09.421679 19665 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786517687384214-19665-0/minicluster-data/ts-0/wal/instance
I20260812 06:59:09.422107 19665 external_mini_cluster.cc:1428] Setting key 35283e080813a5ebc82f8eb5a76cd29e
I20260812 06:59:09.424826 19665 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu
/tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786517687384214-19665-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786517687384214-19665-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786517687384214-19665-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786517687384214-19665-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.52.66:0
--local_ip_for_outbound_sockets=127.19.52.66
--webserver_interface=127.19.52.66
--webserver_port=0
--tserver_master_addrs=127.19.52.126:42821
--builtin_ntp_servers=127.19.52.84:42247
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260812 06:59:09.430598 7376 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:59:09.420108 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.52.65:51439 (local address 127.19.52.126:42821)
0812 06:59:09.420283 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0812 06:59:09.420288 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 06:59:09.422100 (+ 1812us) server_negotiation.cc:407] Connection header received
0812 06:59:09.424094 (+ 1994us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:59:09.424098 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:59:09.424160 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:59:09.424232 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:59:09.425676 (+ 1444us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:59:09.426254 (+ 578us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:59:09.427950 (+ 1696us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:59:09.428370 (+ 420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:59:09.428508 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:59:09.429873 (+ 1365us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:59:09.429880 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:59:09.429884 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 06:59:09.429908 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:59:09.429940 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 06:59:09.429953 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:59:09.430047 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0812 06:59:09.430271 (+ 224us) server_negotiation.cc:299] Negotiation successful
0812 06:59:09.430364 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260812 06:59:09.431420 7549 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:59:09.420700 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.126:42821 (local address 127.19.52.65:51439)
0812 06:59:09.421836 (+ 1136us) negotiation.cc:107] Waiting for socket to connect
0812 06:59:09.421911 (+ 75us) client_negotiation.cc:174] Beginning negotiation
0812 06:59:09.423829 (+ 1918us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:59:09.424448 (+ 619us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:59:09.424460 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:59:09.424631 (+ 171us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:59:09.425472 (+ 841us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:59:09.425498 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:59:09.426412 (+ 914us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:59:09.426419 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:59:09.427787 (+ 1368us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:59:09.427800 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:59:09.428633 (+ 833us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:59:09.428639 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:59:09.428815 (+ 176us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:59:09.429527 (+ 712us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 06:59:09.429528 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:59:09.429574 (+ 46us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 06:59:09.429652 (+ 78us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 06:59:09.430058 (+ 406us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 06:59:09.430067 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 06:59:09.430072 (+ 5us) client_negotiation.cc:769] Sending connection context
0812 06:59:09.430212 (+ 140us) client_negotiation.cc:240] Negotiation successful
0812 06:59:09.430476 (+ 264us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":412,"thread_start_us":140,"threads_started":1}
I20260812 06:59:09.433022 7547 heartbeater.cc:347] Connected to a master server at 127.19.52.126:42821
I20260812 06:59:09.433331 7547 heartbeater.cc:464] Registering TS with master...
I20260812 06:59:09.433904 7547 heartbeater.cc:511] Master 127.19.52.126:42821 requested a full tablet report, sending...
I20260812 06:59:09.435643 7332 ts_manager.cc:194] Registered new tserver with Master: 7e187814a4a7477e8c7c6385cd56a295 (127.19.52.65:45755)
I20260812 06:59:09.437247 7332 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.19.52.65:51439
W20260812 06:59:09.546993 7550 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:59:09.547314 7550 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:59:09.547358 7550 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 06:59:09.551682 7550 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 06:59:09.551764 7550 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 06:59:09.551826 7550 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.52.66
I20260812 06:59:09.557245 7550 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.52.84:42247
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786517687384214-19665-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786517687384214-19665-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.52.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786517687384214-19665-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.19.52.66
--webserver_port=0
--tserver_master_addrs=127.19.52.126:42821
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7550
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.52.66
--log_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786517687384214-19665-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 06:15:54 UTC on ccycloud.quasar-rnfzce.root.comops.site
build id 4581989
I20260812 06:59:09.558548 7550 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:59:09.560045 7550 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 06:59:09.570588 7559 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:59:09.570680 7556 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:59:09.570575 7557 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:59:09.571113 7550 server_base.cc:1034] running on GCE node
I20260812 06:59:09.571703 7550 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:59:09.572405 7550 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:59:09.573609 7550 hybrid_clock.cc:648] HybridClock initialized: now 1786517949573533 us; error 88 us; skew 500 ppm
I20260812 06:59:09.575670 7550 webserver.cc:460] Webserver started at http://127.19.52.66:40029/ using document root <none> and password file <none>
I20260812 06:59:09.576297 7550 fs_manager.cc:359] Metadata directory not provided
I20260812 06:59:09.576399 7550 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:59:09.576728 7550 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 06:59:09.578941 7550 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786517687384214-19665-0/minicluster-data/ts-1/data/instance:
uuid: "e4e1d8ac0d0c43fc91e3cb5808a9f99f"
format_stamp: "Formatted at 2026-08-12 06:59:09 on dist-test-slave-65mx"
server_key: "b2ef23fbfd91a3871c5f1e3aae74f81f"
server_key_iv: "4c0eb39644b2835fca46c693dd3f3b4e"
server_key_version: "encryptionkey@0"
I20260812 06:59:09.579548 7550 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786517687384214-19665-0/minicluster-data/ts-1/wal/instance:
uuid: "e4e1d8ac0d0c43fc91e3cb5808a9f99f"
format_stamp: "Formatted at 2026-08-12 06:59:09 on dist-test-slave-65mx"
server_key: "b2ef23fbfd91a3871c5f1e3aae74f81f"
server_key_iv: "4c0eb39644b2835fca46c693dd3f3b4e"
server_key_version: "encryptionkey@0"
I20260812 06:59:09.583606 7550 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.004s
I20260812 06:59:09.586853 7566 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 06:59:09.588420 7550 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260812 06:59:09.588610 7550 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786517687384214-19665-0/minicluster-data/ts-1/data,/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786517687384214-19665-0/minicluster-data/ts-1/wal
uuid: "e4e1d8ac0d0c43fc91e3cb5808a9f99f"
format_stamp: "Formatted at 2026-08-12 06:59:09 on dist-test-slave-65mx"
server_key: "b2ef23fbfd91a3871c5f1e3aae74f81f"
server_key_iv: "4c0eb39644b2835fca46c693dd3f3b4e"
server_key_version: "encryptionkey@0"
I20260812 06:59:09.588798 7550 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786517687384214-19665-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786517687384214-19665-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786517687384214-19665-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:59:09.623261 7550 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 06:59:09.624070 7550 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 06:59:09.624238 7550 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:59:09.624891 7550 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 06:59:09.625823 7550 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 06:59:09.625870 7550 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 06:59:09.625912 7550 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 06:59:09.625928 7550 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 06:59:09.637459 7550 rpc_server.cc:307] RPC server started. Bound to: 127.19.52.66:33385
I20260812 06:59:09.637562 7693 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.52.66:33385 every 8 connection(s)
I20260812 06:59:09.638528 7550 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786517687384214-19665-0/minicluster-data/ts-1/data/info.pb
I20260812 06:59:09.644239 19665 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu as pid 7550
I20260812 06:59:09.644373 19665 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786517687384214-19665-0/minicluster-data/ts-1/wal/instance
I20260812 06:59:09.644881 19665 external_mini_cluster.cc:1428] Setting key 98c509d1d7bb89ad36753410845ed235
I20260812 06:59:09.647254 19665 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu
/tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786517687384214-19665-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786517687384214-19665-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786517687384214-19665-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786517687384214-19665-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.52.67:0
--local_ip_for_outbound_sockets=127.19.52.67
--webserver_interface=127.19.52.67
--webserver_port=0
--tserver_master_addrs=127.19.52.126:42821
--builtin_ntp_servers=127.19.52.84:42247
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260812 06:59:09.649987 7376 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:59:09.640787 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.52.66:50783 (local address 127.19.52.126:42821)
0812 06:59:09.640943 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0812 06:59:09.640947 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 06:59:09.642311 (+ 1364us) server_negotiation.cc:407] Connection header received
0812 06:59:09.643858 (+ 1547us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:59:09.643861 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:59:09.643931 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:59:09.644048 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:59:09.645284 (+ 1236us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:59:09.645930 (+ 646us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:59:09.647330 (+ 1400us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:59:09.647779 (+ 449us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:59:09.647921 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:59:09.649393 (+ 1472us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:59:09.649399 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:59:09.649402 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 06:59:09.649424 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:59:09.649460 (+ 36us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 06:59:09.649471 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:59:09.649535 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0812 06:59:09.649775 (+ 240us) server_negotiation.cc:299] Negotiation successful
0812 06:59:09.649846 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260812 06:59:09.651274 7696 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:59:09.641224 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.126:42821 (local address 127.19.52.66:50783)
0812 06:59:09.642102 (+ 878us) negotiation.cc:107] Waiting for socket to connect
0812 06:59:09.642151 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0812 06:59:09.643612 (+ 1461us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:59:09.644254 (+ 642us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:59:09.644261 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:59:09.644395 (+ 134us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:59:09.645108 (+ 713us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:59:09.645126 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:59:09.646088 (+ 962us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:59:09.646092 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:59:09.647189 (+ 1097us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:59:09.647199 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:59:09.648061 (+ 862us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:59:09.648066 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:59:09.648236 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:59:09.648988 (+ 752us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 06:59:09.648989 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:59:09.649046 (+ 57us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 06:59:09.649159 (+ 113us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 06:59:09.649567 (+ 408us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 06:59:09.649579 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 06:59:09.649587 (+ 8us) client_negotiation.cc:769] Sending connection context
0812 06:59:09.649744 (+ 157us) client_negotiation.cc:240] Negotiation successful
0812 06:59:09.650145 (+ 401us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":388,"thread_start_us":149,"threads_started":1}
I20260812 06:59:09.652832 7694 heartbeater.cc:347] Connected to a master server at 127.19.52.126:42821
I20260812 06:59:09.653169 7694 heartbeater.cc:464] Registering TS with master...
I20260812 06:59:09.654017 7694 heartbeater.cc:511] Master 127.19.52.126:42821 requested a full tablet report, sending...
I20260812 06:59:09.655519 7332 ts_manager.cc:194] Registered new tserver with Master: e4e1d8ac0d0c43fc91e3cb5808a9f99f (127.19.52.66:33385)
I20260812 06:59:09.656219 7332 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.19.52.66:50783
W20260812 06:59:09.768839 7697 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:59:09.769239 7697 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:59:09.769325 7697 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 06:59:09.773583 7697 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 06:59:09.773675 7697 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 06:59:09.773765 7697 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.52.67
I20260812 06:59:09.778939 7697 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.52.84:42247
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786517687384214-19665-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786517687384214-19665-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.52.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786517687384214-19665-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.19.52.67
--webserver_port=0
--tserver_master_addrs=127.19.52.126:42821
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7697
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.52.67
--log_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786517687384214-19665-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 06:15:54 UTC on ccycloud.quasar-rnfzce.root.comops.site
build id 4581989
I20260812 06:59:09.780200 7697 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:59:09.781757 7697 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 06:59:09.792495 7707 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:59:09.792707 7703 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:59:09.792495 7704 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:59:09.793007 7697 server_base.cc:1034] running on GCE node
I20260812 06:59:09.793603 7697 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:59:09.794360 7697 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:59:09.795593 7697 hybrid_clock.cc:648] HybridClock initialized: now 1786517949795553 us; error 54 us; skew 500 ppm
I20260812 06:59:09.797825 7697 webserver.cc:460] Webserver started at http://127.19.52.67:32951/ using document root <none> and password file <none>
I20260812 06:59:09.798446 7697 fs_manager.cc:359] Metadata directory not provided
I20260812 06:59:09.798548 7697 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:59:09.798808 7697 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 06:59:09.801177 7697 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786517687384214-19665-0/minicluster-data/ts-2/data/instance:
uuid: "917bef5472ff4182a74f83edc0884017"
format_stamp: "Formatted at 2026-08-12 06:59:09 on dist-test-slave-65mx"
server_key: "49b57533b86e476fc7f2c15d8b17404b"
server_key_iv: "e4a5abb79ba720f987673f57cdf18d56"
server_key_version: "encryptionkey@0"
I20260812 06:59:09.801801 7697 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786517687384214-19665-0/minicluster-data/ts-2/wal/instance:
uuid: "917bef5472ff4182a74f83edc0884017"
format_stamp: "Formatted at 2026-08-12 06:59:09 on dist-test-slave-65mx"
server_key: "49b57533b86e476fc7f2c15d8b17404b"
server_key_iv: "e4a5abb79ba720f987673f57cdf18d56"
server_key_version: "encryptionkey@0"
I20260812 06:59:09.806336 7697 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260812 06:59:09.809406 7713 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 06:59:09.810920 7697 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.004s
I20260812 06:59:09.811101 7697 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786517687384214-19665-0/minicluster-data/ts-2/data,/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786517687384214-19665-0/minicluster-data/ts-2/wal
uuid: "917bef5472ff4182a74f83edc0884017"
format_stamp: "Formatted at 2026-08-12 06:59:09 on dist-test-slave-65mx"
server_key: "49b57533b86e476fc7f2c15d8b17404b"
server_key_iv: "e4a5abb79ba720f987673f57cdf18d56"
server_key_version: "encryptionkey@0"
I20260812 06:59:09.811240 7697 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786517687384214-19665-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786517687384214-19665-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786517687384214-19665-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:59:09.844658 7697 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 06:59:09.845475 7697 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 06:59:09.845701 7697 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:59:09.846354 7697 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 06:59:09.847389 7697 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 06:59:09.847467 7697 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 06:59:09.847546 7697 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 06:59:09.847595 7697 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 06:59:09.857280 7697 rpc_server.cc:307] RPC server started. Bound to: 127.19.52.67:39669
I20260812 06:59:09.857359 7842 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.52.67:39669 every 8 connection(s)
I20260812 06:59:09.858285 7697 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786517687384214-19665-0/minicluster-data/ts-2/data/info.pb
I20260812 06:59:09.867261 19665 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu as pid 7697
I20260812 06:59:09.867389 19665 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786517687384214-19665-0/minicluster-data/ts-2/wal/instance
I20260812 06:59:09.867763 19665 external_mini_cluster.cc:1428] Setting key 639f5f1992446d45edd8eb77a13d6a61
I20260812 06:59:09.869174 7376 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:59:09.860375 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.52.67:43177 (local address 127.19.52.126:42821)
0812 06:59:09.860630 (+ 255us) server_negotiation.cc:206] Beginning negotiation
0812 06:59:09.860635 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 06:59:09.861743 (+ 1108us) server_negotiation.cc:407] Connection header received
0812 06:59:09.863291 (+ 1548us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:59:09.863294 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:59:09.863350 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:59:09.863438 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:59:09.864534 (+ 1096us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:59:09.865177 (+ 643us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:59:09.866562 (+ 1385us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:59:09.866806 (+ 244us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:59:09.866910 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:59:09.868584 (+ 1674us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:59:09.868588 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:59:09.868591 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 06:59:09.868607 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:59:09.868633 (+ 26us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 06:59:09.868647 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:59:09.868728 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0812 06:59:09.868907 (+ 179us) server_negotiation.cc:299] Negotiation successful
0812 06:59:09.868982 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":90}
I20260812 06:59:09.870260 7845 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:59:09.860781 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.126:42821 (local address 127.19.52.67:43177)
0812 06:59:09.861572 (+ 791us) negotiation.cc:107] Waiting for socket to connect
0812 06:59:09.861615 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0812 06:59:09.863027 (+ 1412us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:59:09.863680 (+ 653us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:59:09.863688 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:59:09.863787 (+ 99us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:59:09.864322 (+ 535us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:59:09.864336 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:59:09.865360 (+ 1024us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:59:09.865363 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:59:09.866376 (+ 1013us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:59:09.866386 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:59:09.867053 (+ 667us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:59:09.867061 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:59:09.867309 (+ 248us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:59:09.868182 (+ 873us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 06:59:09.868184 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:59:09.868252 (+ 68us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 06:59:09.868359 (+ 107us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 06:59:09.868731 (+ 372us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 06:59:09.868742 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 06:59:09.868751 (+ 9us) client_negotiation.cc:769] Sending connection context
0812 06:59:09.868899 (+ 148us) client_negotiation.cc:240] Negotiation successful
0812 06:59:09.869223 (+ 324us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":378,"thread_start_us":144,"threads_started":1}
I20260812 06:59:09.871652 7843 heartbeater.cc:347] Connected to a master server at 127.19.52.126:42821
I20260812 06:59:09.871925 7843 heartbeater.cc:464] Registering TS with master...
I20260812 06:59:09.872531 7843 heartbeater.cc:511] Master 127.19.52.126:42821 requested a full tablet report, sending...
I20260812 06:59:09.873687 7332 ts_manager.cc:194] Registered new tserver with Master: 917bef5472ff4182a74f83edc0884017 (127.19.52.67:39669)
I20260812 06:59:09.874284 7332 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.19.52.67:43177
I20260812 06:59:09.882575 19665 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 06:59:09.889683 7376 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:59:09.884927 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33468 (local address 127.19.52.126:42821)
0812 06:59:09.885148 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0812 06:59:09.885152 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 06:59:09.885314 (+ 162us) server_negotiation.cc:407] Connection header received
0812 06:59:09.885436 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:59:09.885441 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:59:09.885592 (+ 151us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:59:09.885673 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:59:09.886048 (+ 375us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:59:09.886760 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:59:09.887808 (+ 1048us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:59:09.888029 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:59:09.888246 (+ 217us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:59:09.889168 (+ 922us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:59:09.889171 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:59:09.889173 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 06:59:09.889187 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:59:09.889206 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 06:59:09.889215 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:59:09.889282 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0812 06:59:09.889400 (+ 118us) server_negotiation.cc:299] Negotiation successful
0812 06:59:09.889466 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":117}
I20260812 06:59:09.893087 7332 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:33468:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 06:59:09.896310 7332 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 06:59:09.912832 7860 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:59:09.907111 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50238 (local address 127.19.52.66:33385)
0812 06:59:09.907743 (+ 632us) server_negotiation.cc:206] Beginning negotiation
0812 06:59:09.907757 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0812 06:59:09.907786 (+ 29us) server_negotiation.cc:407] Connection header received
0812 06:59:09.907872 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:59:09.907879 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:59:09.908125 (+ 246us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:59:09.908289 (+ 164us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 06:59:09.908671 (+ 382us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:59:09.909525 (+ 854us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:59:09.911100 (+ 1575us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:59:09.911933 (+ 833us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:59:09.912063 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:59:09.912293 (+ 230us) server_negotiation.cc:1035] Waiting for connection context
0812 06:59:09.912472 (+ 179us) server_negotiation.cc:299] Negotiation successful
0812 06:59:09.912630 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":513,"thread_start_us":139,"threads_started":1}
I20260812 06:59:09.913225 7856 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:59:09.906702 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.66:33385 (local address 127.0.0.1:50238)
0812 06:59:09.907688 (+ 986us) negotiation.cc:107] Waiting for socket to connect
0812 06:59:09.907698 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0812 06:59:09.907767 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:59:09.908301 (+ 534us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:59:09.908307 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:59:09.908333 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 06:59:09.908520 (+ 187us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:59:09.908529 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:59:09.909654 (+ 1125us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:59:09.909661 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:59:09.910942 (+ 1281us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:59:09.910953 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:59:09.912085 (+ 1132us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:59:09.912088 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:59:09.912228 (+ 140us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:59:09.912274 (+ 46us) client_negotiation.cc:769] Sending connection context
0812 06:59:09.912421 (+ 147us) client_negotiation.cc:240] Negotiation successful
0812 06:59:09.912751 (+ 330us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":936,"thread_start_us":99,"threads_started":1}
I20260812 06:59:09.913269 7859 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:59:09.907131 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59556 (local address 127.19.52.65:45755)
0812 06:59:09.908039 (+ 908us) server_negotiation.cc:206] Beginning negotiation
0812 06:59:09.908045 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 06:59:09.908070 (+ 25us) server_negotiation.cc:407] Connection header received
0812 06:59:09.908163 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:59:09.908174 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:59:09.908380 (+ 206us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:59:09.908539 (+ 159us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 06:59:09.909051 (+ 512us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:59:09.910144 (+ 1093us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:59:09.911465 (+ 1321us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:59:09.912245 (+ 780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:59:09.912355 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:59:09.912713 (+ 358us) server_negotiation.cc:1035] Waiting for connection context
0812 06:59:09.912772 (+ 59us) server_negotiation.cc:299] Negotiation successful
0812 06:59:09.912867 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":796,"thread_start_us":122,"threads_started":1}
I20260812 06:59:09.913408 7854 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:59:09.906209 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.67:39669 (local address 127.0.0.1:56256)
0812 06:59:09.907111 (+ 902us) negotiation.cc:107] Waiting for socket to connect
0812 06:59:09.907172 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0812 06:59:09.907326 (+ 154us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:59:09.908485 (+ 1159us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:59:09.908488 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:59:09.908505 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 06:59:09.908632 (+ 127us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:59:09.908639 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:59:09.910159 (+ 1520us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:59:09.910162 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:59:09.911249 (+ 1087us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:59:09.911258 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:59:09.912935 (+ 1677us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:59:09.912939 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:59:09.913095 (+ 156us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:59:09.913119 (+ 24us) client_negotiation.cc:769] Sending connection context
0812 06:59:09.913205 (+ 86us) client_negotiation.cc:240] Negotiation successful
0812 06:59:09.913278 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":723,"thread_start_us":109,"threads_started":1}
I20260812 06:59:09.913790 7857 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:59:09.906452 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56256 (local address 127.19.52.67:39669)
0812 06:59:09.908041 (+ 1589us) server_negotiation.cc:206] Beginning negotiation
0812 06:59:09.908052 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0812 06:59:09.908075 (+ 23us) server_negotiation.cc:407] Connection header received
0812 06:59:09.908148 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:59:09.908153 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:59:09.908320 (+ 167us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:59:09.908460 (+ 140us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 06:59:09.908801 (+ 341us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:59:09.910016 (+ 1215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:59:09.911381 (+ 1365us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:59:09.911946 (+ 565us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:59:09.912086 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:59:09.912288 (+ 202us) server_negotiation.cc:1035] Waiting for connection context
0812 06:59:09.913266 (+ 978us) server_negotiation.cc:299] Negotiation successful
0812 06:59:09.913394 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1484,"thread_start_us":170,"threads_started":1}
I20260812 06:59:09.913798 7858 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:59:09.906462 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.65:45755 (local address 127.0.0.1:59556)
0812 06:59:09.907217 (+ 755us) negotiation.cc:107] Waiting for socket to connect
0812 06:59:09.907232 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 06:59:09.907333 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:59:09.908637 (+ 1304us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:59:09.908639 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:59:09.908660 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 06:59:09.908802 (+ 142us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:59:09.908809 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:59:09.910275 (+ 1466us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:59:09.910281 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:59:09.911293 (+ 1012us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:59:09.911302 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:59:09.912373 (+ 1071us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:59:09.912377 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:59:09.912608 (+ 231us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:59:09.912635 (+ 27us) client_negotiation.cc:769] Sending connection context
0812 06:59:09.912725 (+ 90us) client_negotiation.cc:240] Negotiation successful
0812 06:59:09.912805 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":678,"thread_start_us":113,"threads_started":1}
I20260812 06:59:09.916854 7769 tablet_service.cc:1467] Processing CreateTablet for tablet 2e3e21ae7a9d47a28cde04334c41b44b (DEFAULT_TABLE table=test-table [id=761f7249f1c4489b94b1a3b1a6e81b6f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 06:59:09.916863 7454 tablet_service.cc:1467] Processing CreateTablet for tablet 2e3e21ae7a9d47a28cde04334c41b44b (DEFAULT_TABLE table=test-table [id=761f7249f1c4489b94b1a3b1a6e81b6f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 06:59:09.916864 7617 tablet_service.cc:1467] Processing CreateTablet for tablet 2e3e21ae7a9d47a28cde04334c41b44b (DEFAULT_TABLE table=test-table [id=761f7249f1c4489b94b1a3b1a6e81b6f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 06:59:09.917927 7769 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2e3e21ae7a9d47a28cde04334c41b44b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 06:59:09.917953 7454 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2e3e21ae7a9d47a28cde04334c41b44b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 06:59:09.917953 7617 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2e3e21ae7a9d47a28cde04334c41b44b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 06:59:09.924710 7861 tablet_bootstrap.cc:492] T 2e3e21ae7a9d47a28cde04334c41b44b P 917bef5472ff4182a74f83edc0884017: Bootstrap starting.
I20260812 06:59:09.925405 7862 tablet_bootstrap.cc:492] T 2e3e21ae7a9d47a28cde04334c41b44b P e4e1d8ac0d0c43fc91e3cb5808a9f99f: Bootstrap starting.
I20260812 06:59:09.926055 7864 tablet_bootstrap.cc:492] T 2e3e21ae7a9d47a28cde04334c41b44b P 7e187814a4a7477e8c7c6385cd56a295: Bootstrap starting.
I20260812 06:59:09.927742 7864 tablet_bootstrap.cc:654] T 2e3e21ae7a9d47a28cde04334c41b44b P 7e187814a4a7477e8c7c6385cd56a295: Neither blocks nor log segments found. Creating new log.
I20260812 06:59:09.927763 7862 tablet_bootstrap.cc:654] T 2e3e21ae7a9d47a28cde04334c41b44b P e4e1d8ac0d0c43fc91e3cb5808a9f99f: Neither blocks nor log segments found. Creating new log.
I20260812 06:59:09.927762 7861 tablet_bootstrap.cc:654] T 2e3e21ae7a9d47a28cde04334c41b44b P 917bef5472ff4182a74f83edc0884017: Neither blocks nor log segments found. Creating new log.
I20260812 06:59:09.928501 7864 log.cc:824] T 2e3e21ae7a9d47a28cde04334c41b44b P 7e187814a4a7477e8c7c6385cd56a295: Log is configured to *not* fsync() on all Append() calls
I20260812 06:59:09.928620 7862 log.cc:824] T 2e3e21ae7a9d47a28cde04334c41b44b P e4e1d8ac0d0c43fc91e3cb5808a9f99f: Log is configured to *not* fsync() on all Append() calls
I20260812 06:59:09.928620 7861 log.cc:824] T 2e3e21ae7a9d47a28cde04334c41b44b P 917bef5472ff4182a74f83edc0884017: Log is configured to *not* fsync() on all Append() calls
I20260812 06:59:09.930751 7861 tablet_bootstrap.cc:492] T 2e3e21ae7a9d47a28cde04334c41b44b P 917bef5472ff4182a74f83edc0884017: No bootstrap required, opened a new log
I20260812 06:59:09.930835 7864 tablet_bootstrap.cc:492] T 2e3e21ae7a9d47a28cde04334c41b44b P 7e187814a4a7477e8c7c6385cd56a295: No bootstrap required, opened a new log
I20260812 06:59:09.930935 7862 tablet_bootstrap.cc:492] T 2e3e21ae7a9d47a28cde04334c41b44b P e4e1d8ac0d0c43fc91e3cb5808a9f99f: No bootstrap required, opened a new log
I20260812 06:59:09.930999 7861 ts_tablet_manager.cc:1397] T 2e3e21ae7a9d47a28cde04334c41b44b P 917bef5472ff4182a74f83edc0884017: Time spent bootstrapping tablet: real 0.007s user 0.004s sys 0.000s
I20260812 06:59:09.931026 7864 ts_tablet_manager.cc:1397] T 2e3e21ae7a9d47a28cde04334c41b44b P 7e187814a4a7477e8c7c6385cd56a295: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260812 06:59:09.931087 7862 ts_tablet_manager.cc:1397] T 2e3e21ae7a9d47a28cde04334c41b44b P e4e1d8ac0d0c43fc91e3cb5808a9f99f: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260812 06:59:09.934725 7861 raft_consensus.cc:348] T 2e3e21ae7a9d47a28cde04334c41b44b P 917bef5472ff4182a74f83edc0884017 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "917bef5472ff4182a74f83edc0884017" member_type: VOTER last_known_addr { host: "127.19.52.67" port: 39669 } } peers { permanent_uuid: "7e187814a4a7477e8c7c6385cd56a295" member_type: VOTER last_known_addr { host: "127.19.52.65" port: 45755 } } peers { permanent_uuid: "e4e1d8ac0d0c43fc91e3cb5808a9f99f" member_type: VOTER last_known_addr { host: "127.19.52.66" port: 33385 } }
I20260812 06:59:09.935132 7861 raft_consensus.cc:374] T 2e3e21ae7a9d47a28cde04334c41b44b P 917bef5472ff4182a74f83edc0884017 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 06:59:09.935191 7861 raft_consensus.cc:729] T 2e3e21ae7a9d47a28cde04334c41b44b P 917bef5472ff4182a74f83edc0884017 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 917bef5472ff4182a74f83edc0884017, State: Initialized, Role: FOLLOWER
I20260812 06:59:09.935321 7862 raft_consensus.cc:348] T 2e3e21ae7a9d47a28cde04334c41b44b P e4e1d8ac0d0c43fc91e3cb5808a9f99f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "917bef5472ff4182a74f83edc0884017" member_type: VOTER last_known_addr { host: "127.19.52.67" port: 39669 } } peers { permanent_uuid: "7e187814a4a7477e8c7c6385cd56a295" member_type: VOTER last_known_addr { host: "127.19.52.65" port: 45755 } } peers { permanent_uuid: "e4e1d8ac0d0c43fc91e3cb5808a9f99f" member_type: VOTER last_known_addr { host: "127.19.52.66" port: 33385 } }
I20260812 06:59:09.935321 7864 raft_consensus.cc:348] T 2e3e21ae7a9d47a28cde04334c41b44b P 7e187814a4a7477e8c7c6385cd56a295 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "917bef5472ff4182a74f83edc0884017" member_type: VOTER last_known_addr { host: "127.19.52.67" port: 39669 } } peers { permanent_uuid: "7e187814a4a7477e8c7c6385cd56a295" member_type: VOTER last_known_addr { host: "127.19.52.65" port: 45755 } } peers { permanent_uuid: "e4e1d8ac0d0c43fc91e3cb5808a9f99f" member_type: VOTER last_known_addr { host: "127.19.52.66" port: 33385 } }
I20260812 06:59:09.935578 7864 raft_consensus.cc:374] T 2e3e21ae7a9d47a28cde04334c41b44b P 7e187814a4a7477e8c7c6385cd56a295 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 06:59:09.935578 7862 raft_consensus.cc:374] T 2e3e21ae7a9d47a28cde04334c41b44b P e4e1d8ac0d0c43fc91e3cb5808a9f99f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 06:59:09.935638 7864 raft_consensus.cc:729] T 2e3e21ae7a9d47a28cde04334c41b44b P 7e187814a4a7477e8c7c6385cd56a295 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7e187814a4a7477e8c7c6385cd56a295, State: Initialized, Role: FOLLOWER
I20260812 06:59:09.935649 7862 raft_consensus.cc:729] T 2e3e21ae7a9d47a28cde04334c41b44b P e4e1d8ac0d0c43fc91e3cb5808a9f99f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e4e1d8ac0d0c43fc91e3cb5808a9f99f, State: Initialized, Role: FOLLOWER
I20260812 06:59:09.935691 7861 consensus_queue.cc:260] T 2e3e21ae7a9d47a28cde04334c41b44b P 917bef5472ff4182a74f83edc0884017 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "917bef5472ff4182a74f83edc0884017" member_type: VOTER last_known_addr { host: "127.19.52.67" port: 39669 } } peers { permanent_uuid: "7e187814a4a7477e8c7c6385cd56a295" member_type: VOTER last_known_addr { host: "127.19.52.65" port: 45755 } } peers { permanent_uuid: "e4e1d8ac0d0c43fc91e3cb5808a9f99f" member_type: VOTER last_known_addr { host: "127.19.52.66" port: 33385 } }
I20260812 06:59:09.936108 7864 consensus_queue.cc:260] T 2e3e21ae7a9d47a28cde04334c41b44b P 7e187814a4a7477e8c7c6385cd56a295 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "917bef5472ff4182a74f83edc0884017" member_type: VOTER last_known_addr { host: "127.19.52.67" port: 39669 } } peers { permanent_uuid: "7e187814a4a7477e8c7c6385cd56a295" member_type: VOTER last_known_addr { host: "127.19.52.65" port: 45755 } } peers { permanent_uuid: "e4e1d8ac0d0c43fc91e3cb5808a9f99f" member_type: VOTER last_known_addr { host: "127.19.52.66" port: 33385 } }
I20260812 06:59:09.936259 7862 consensus_queue.cc:260] T 2e3e21ae7a9d47a28cde04334c41b44b P e4e1d8ac0d0c43fc91e3cb5808a9f99f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "917bef5472ff4182a74f83edc0884017" member_type: VOTER last_known_addr { host: "127.19.52.67" port: 39669 } } peers { permanent_uuid: "7e187814a4a7477e8c7c6385cd56a295" member_type: VOTER last_known_addr { host: "127.19.52.65" port: 45755 } } peers { permanent_uuid: "e4e1d8ac0d0c43fc91e3cb5808a9f99f" member_type: VOTER last_known_addr { host: "127.19.52.66" port: 33385 } }
I20260812 06:59:09.936949 7861 ts_tablet_manager.cc:1428] T 2e3e21ae7a9d47a28cde04334c41b44b P 917bef5472ff4182a74f83edc0884017: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260812 06:59:09.937152 7864 ts_tablet_manager.cc:1428] T 2e3e21ae7a9d47a28cde04334c41b44b P 7e187814a4a7477e8c7c6385cd56a295: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260812 06:59:09.937144 7862 ts_tablet_manager.cc:1428] T 2e3e21ae7a9d47a28cde04334c41b44b P e4e1d8ac0d0c43fc91e3cb5808a9f99f: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260812 06:59:09.937351 7843 heartbeater.cc:503] Master 127.19.52.126:42821 was elected leader, sending a full tablet report...
I20260812 06:59:09.937489 7694 heartbeater.cc:503] Master 127.19.52.126:42821 was elected leader, sending a full tablet report...
I20260812 06:59:09.937496 7547 heartbeater.cc:503] Master 127.19.52.126:42821 was elected leader, sending a full tablet report...
I20260812 06:59:10.043131 7872 raft_consensus.cc:482] T 2e3e21ae7a9d47a28cde04334c41b44b P e4e1d8ac0d0c43fc91e3cb5808a9f99f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 06:59:10.043394 7872 raft_consensus.cc:504] T 2e3e21ae7a9d47a28cde04334c41b44b P e4e1d8ac0d0c43fc91e3cb5808a9f99f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "917bef5472ff4182a74f83edc0884017" member_type: VOTER last_known_addr { host: "127.19.52.67" port: 39669 } } peers { permanent_uuid: "7e187814a4a7477e8c7c6385cd56a295" member_type: VOTER last_known_addr { host: "127.19.52.65" port: 45755 } } peers { permanent_uuid: "e4e1d8ac0d0c43fc91e3cb5808a9f99f" member_type: VOTER last_known_addr { host: "127.19.52.66" port: 33385 } }
I20260812 06:59:10.044762 7872 leader_election.cc:288] T 2e3e21ae7a9d47a28cde04334c41b44b P e4e1d8ac0d0c43fc91e3cb5808a9f99f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 917bef5472ff4182a74f83edc0884017 (127.19.52.67:39669), 7e187814a4a7477e8c7c6385cd56a295 (127.19.52.65:45755)
I20260812 06:59:10.049095 7857 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:59:10.045156 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.52.66:51581 (local address 127.19.52.67:39669)
0812 06:59:10.045343 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0812 06:59:10.045349 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 06:59:10.045367 (+ 18us) server_negotiation.cc:407] Connection header received
0812 06:59:10.045434 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:59:10.045437 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:59:10.045491 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:59:10.045582 (+ 91us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 06:59:10.046058 (+ 476us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:59:10.046706 (+ 648us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:59:10.047986 (+ 1280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:59:10.048421 (+ 435us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:59:10.048599 (+ 178us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:59:10.048667 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0812 06:59:10.048873 (+ 206us) server_negotiation.cc:299] Negotiation successful
0812 06:59:10.048972 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":88}
I20260812 06:59:10.049095 7696 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:59:10.044974 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.67:39669 (local address 127.19.52.66:51581)
0812 06:59:10.045138 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0812 06:59:10.045156 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0812 06:59:10.045272 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:59:10.045693 (+ 421us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:59:10.045700 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:59:10.045760 (+ 60us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 06:59:10.045908 (+ 148us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:59:10.045920 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:59:10.046845 (+ 925us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:59:10.046849 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:59:10.047830 (+ 981us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:59:10.047841 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:59:10.048597 (+ 756us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:59:10.048601 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:59:10.048715 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:59:10.048743 (+ 28us) client_negotiation.cc:769] Sending connection context
0812 06:59:10.048840 (+ 97us) client_negotiation.cc:240] Negotiation successful
0812 06:59:10.048932 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":57}
I20260812 06:59:10.049577 7873 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:59:10.045154 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.65:45755 (local address 127.19.52.66:42415)
0812 06:59:10.045505 (+ 351us) negotiation.cc:107] Waiting for socket to connect
0812 06:59:10.045519 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 06:59:10.045691 (+ 172us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:59:10.046052 (+ 361us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:59:10.046055 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:59:10.046075 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 06:59:10.046188 (+ 113us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:59:10.046194 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:59:10.047266 (+ 1072us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:59:10.047269 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:59:10.048123 (+ 854us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:59:10.048134 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:59:10.048933 (+ 799us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:59:10.048937 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:59:10.049047 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:59:10.049079 (+ 32us) client_negotiation.cc:769] Sending connection context
0812 06:59:10.049345 (+ 266us) client_negotiation.cc:240] Negotiation successful
0812 06:59:10.049428 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":286,"thread_start_us":137,"threads_started":1}
I20260812 06:59:10.049600 7859 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:59:10.045342 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.52.66:42415 (local address 127.19.52.65:45755)
0812 06:59:10.045489 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0812 06:59:10.045493 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 06:59:10.045669 (+ 176us) server_negotiation.cc:407] Connection header received
0812 06:59:10.045822 (+ 153us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:59:10.045825 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:59:10.045891 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:59:10.046002 (+ 111us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 06:59:10.046343 (+ 341us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:59:10.047130 (+ 787us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:59:10.048275 (+ 1145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:59:10.048774 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:59:10.048933 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:59:10.049008 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0812 06:59:10.049376 (+ 368us) server_negotiation.cc:299] Negotiation successful
0812 06:59:10.049462 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260812 06:59:10.049983 7791 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2e3e21ae7a9d47a28cde04334c41b44b" candidate_uuid: "e4e1d8ac0d0c43fc91e3cb5808a9f99f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "917bef5472ff4182a74f83edc0884017" is_pre_election: true
I20260812 06:59:10.050336 7491 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2e3e21ae7a9d47a28cde04334c41b44b" candidate_uuid: "e4e1d8ac0d0c43fc91e3cb5808a9f99f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7e187814a4a7477e8c7c6385cd56a295" is_pre_election: true
I20260812 06:59:10.050374 7791 raft_consensus.cc:2454] T 2e3e21ae7a9d47a28cde04334c41b44b P 917bef5472ff4182a74f83edc0884017 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e4e1d8ac0d0c43fc91e3cb5808a9f99f in term 0.
I20260812 06:59:10.050608 7491 raft_consensus.cc:2454] T 2e3e21ae7a9d47a28cde04334c41b44b P 7e187814a4a7477e8c7c6385cd56a295 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e4e1d8ac0d0c43fc91e3cb5808a9f99f in term 0.
I20260812 06:59:10.051018 7570 leader_election.cc:302] T 2e3e21ae7a9d47a28cde04334c41b44b P e4e1d8ac0d0c43fc91e3cb5808a9f99f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 917bef5472ff4182a74f83edc0884017, e4e1d8ac0d0c43fc91e3cb5808a9f99f; no voters:
I20260812 06:59:10.051383 7872 raft_consensus.cc:2781] T 2e3e21ae7a9d47a28cde04334c41b44b P e4e1d8ac0d0c43fc91e3cb5808a9f99f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 06:59:10.051481 7872 raft_consensus.cc:482] T 2e3e21ae7a9d47a28cde04334c41b44b P e4e1d8ac0d0c43fc91e3cb5808a9f99f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 06:59:10.051528 7872 raft_consensus.cc:3037] T 2e3e21ae7a9d47a28cde04334c41b44b P e4e1d8ac0d0c43fc91e3cb5808a9f99f [term 0 FOLLOWER]: Advancing to term 1
I20260812 06:59:10.052733 7872 raft_consensus.cc:504] T 2e3e21ae7a9d47a28cde04334c41b44b P e4e1d8ac0d0c43fc91e3cb5808a9f99f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "917bef5472ff4182a74f83edc0884017" member_type: VOTER last_known_addr { host: "127.19.52.67" port: 39669 } } peers { permanent_uuid: "7e187814a4a7477e8c7c6385cd56a295" member_type: VOTER last_known_addr { host: "127.19.52.65" port: 45755 } } peers { permanent_uuid: "e4e1d8ac0d0c43fc91e3cb5808a9f99f" member_type: VOTER last_known_addr { host: "127.19.52.66" port: 33385 } }
I20260812 06:59:10.053138 7872 leader_election.cc:288] T 2e3e21ae7a9d47a28cde04334c41b44b P e4e1d8ac0d0c43fc91e3cb5808a9f99f [CANDIDATE]: Term 1 election: Requested vote from peers 917bef5472ff4182a74f83edc0884017 (127.19.52.67:39669), 7e187814a4a7477e8c7c6385cd56a295 (127.19.52.65:45755)
I20260812 06:59:10.053612 7791 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2e3e21ae7a9d47a28cde04334c41b44b" candidate_uuid: "e4e1d8ac0d0c43fc91e3cb5808a9f99f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "917bef5472ff4182a74f83edc0884017"
I20260812 06:59:10.053642 7491 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2e3e21ae7a9d47a28cde04334c41b44b" candidate_uuid: "e4e1d8ac0d0c43fc91e3cb5808a9f99f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7e187814a4a7477e8c7c6385cd56a295"
I20260812 06:59:10.053777 7791 raft_consensus.cc:3037] T 2e3e21ae7a9d47a28cde04334c41b44b P 917bef5472ff4182a74f83edc0884017 [term 0 FOLLOWER]: Advancing to term 1
I20260812 06:59:10.053777 7491 raft_consensus.cc:3037] T 2e3e21ae7a9d47a28cde04334c41b44b P 7e187814a4a7477e8c7c6385cd56a295 [term 0 FOLLOWER]: Advancing to term 1
I20260812 06:59:10.055310 7791 raft_consensus.cc:2454] T 2e3e21ae7a9d47a28cde04334c41b44b P 917bef5472ff4182a74f83edc0884017 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e4e1d8ac0d0c43fc91e3cb5808a9f99f in term 1.
I20260812 06:59:10.055310 7491 raft_consensus.cc:2454] T 2e3e21ae7a9d47a28cde04334c41b44b P 7e187814a4a7477e8c7c6385cd56a295 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e4e1d8ac0d0c43fc91e3cb5808a9f99f in term 1.
I20260812 06:59:10.055747 7569 leader_election.cc:302] T 2e3e21ae7a9d47a28cde04334c41b44b P e4e1d8ac0d0c43fc91e3cb5808a9f99f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7e187814a4a7477e8c7c6385cd56a295, e4e1d8ac0d0c43fc91e3cb5808a9f99f; no voters:
I20260812 06:59:10.055967 7872 raft_consensus.cc:2781] T 2e3e21ae7a9d47a28cde04334c41b44b P e4e1d8ac0d0c43fc91e3cb5808a9f99f [term 1 FOLLOWER]: Leader election won for term 1
I20260812 06:59:10.056265 7872 raft_consensus.cc:686] T 2e3e21ae7a9d47a28cde04334c41b44b P e4e1d8ac0d0c43fc91e3cb5808a9f99f [term 1 LEADER]: Becoming Leader. State: Replica: e4e1d8ac0d0c43fc91e3cb5808a9f99f, State: Running, Role: LEADER
I20260812 06:59:10.056818 7872 consensus_queue.cc:237] T 2e3e21ae7a9d47a28cde04334c41b44b P e4e1d8ac0d0c43fc91e3cb5808a9f99f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "917bef5472ff4182a74f83edc0884017" member_type: VOTER last_known_addr { host: "127.19.52.67" port: 39669 } } peers { permanent_uuid: "7e187814a4a7477e8c7c6385cd56a295" member_type: VOTER last_known_addr { host: "127.19.52.65" port: 45755 } } peers { permanent_uuid: "e4e1d8ac0d0c43fc91e3cb5808a9f99f" member_type: VOTER last_known_addr { host: "127.19.52.66" port: 33385 } }
I20260812 06:59:10.060643 7329 catalog_manager.cc:5495] T 2e3e21ae7a9d47a28cde04334c41b44b P e4e1d8ac0d0c43fc91e3cb5808a9f99f reported cstate change: term changed from 0 to 1, leader changed from <none> to e4e1d8ac0d0c43fc91e3cb5808a9f99f (127.19.52.66). New cstate: current_term: 1 leader_uuid: "e4e1d8ac0d0c43fc91e3cb5808a9f99f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "917bef5472ff4182a74f83edc0884017" member_type: VOTER last_known_addr { host: "127.19.52.67" port: 39669 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7e187814a4a7477e8c7c6385cd56a295" member_type: VOTER last_known_addr { host: "127.19.52.65" port: 45755 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e4e1d8ac0d0c43fc91e3cb5808a9f99f" member_type: VOTER last_known_addr { host: "127.19.52.66" port: 33385 } health_report { overall_health: HEALTHY } } }
I20260812 06:59:10.106668 7860 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:59:10.103018 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50252 (local address 127.19.52.66:33385)
0812 06:59:10.103163 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0812 06:59:10.103167 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 06:59:10.103181 (+ 14us) server_negotiation.cc:407] Connection header received
0812 06:59:10.103261 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:59:10.103263 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:59:10.103321 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:59:10.103411 (+ 90us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 06:59:10.103639 (+ 228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:59:10.104163 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:59:10.105156 (+ 993us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:59:10.105380 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:59:10.105599 (+ 219us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:59:10.105886 (+ 287us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 06:59:10.106335 (+ 449us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 06:59:10.106433 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0812 06:59:10.106495 (+ 62us) server_negotiation.cc:299] Negotiation successful
0812 06:59:10.106549 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
W20260812 06:59:10.109333 7844 tablet.cc:2369] T 2e3e21ae7a9d47a28cde04334c41b44b P 917bef5472ff4182a74f83edc0884017: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 06:59:10.113065 7791 raft_consensus.cc:1261] T 2e3e21ae7a9d47a28cde04334c41b44b P 917bef5472ff4182a74f83edc0884017 [term 1 FOLLOWER]: Refusing update from remote peer e4e1d8ac0d0c43fc91e3cb5808a9f99f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 06:59:10.113252 7491 raft_consensus.cc:1261] T 2e3e21ae7a9d47a28cde04334c41b44b P 7e187814a4a7477e8c7c6385cd56a295 [term 1 FOLLOWER]: Refusing update from remote peer e4e1d8ac0d0c43fc91e3cb5808a9f99f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 06:59:10.114019 7874 consensus_queue.cc:1035] T 2e3e21ae7a9d47a28cde04334c41b44b P e4e1d8ac0d0c43fc91e3cb5808a9f99f [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e187814a4a7477e8c7c6385cd56a295" member_type: VOTER last_known_addr { host: "127.19.52.65" port: 45755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 06:59:10.114301 7872 consensus_queue.cc:1035] T 2e3e21ae7a9d47a28cde04334c41b44b P e4e1d8ac0d0c43fc91e3cb5808a9f99f [LEADER]: Connected to new peer: Peer: permanent_uuid: "917bef5472ff4182a74f83edc0884017" member_type: VOTER last_known_addr { host: "127.19.52.67" port: 39669 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 06:59:10.125474 7877 mvcc.cc:205] Tried to move back new op lower bound from 7317577523654914048 to 7317577523435614208. Current Snapshot: MvccSnapshot[applied={T|T < 7317577523654914048 or (T in {7317577523654914048})}]
I20260812 06:59:10.132498 7329 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:33468:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 06:59:10.132740 7329 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:33468:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 06:59:10.135412 7329 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 7f03d44499c24625ab8abadf93b87969: Sending DeleteTablet for 3 replicas of tablet 2e3e21ae7a9d47a28cde04334c41b44b
I20260812 06:59:10.136282 7769 tablet_service.cc:1514] Processing DeleteTablet for tablet 2e3e21ae7a9d47a28cde04334c41b44b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 06:59:10 UTC) from {username='slave'} at 127.0.0.1:56256
I20260812 06:59:10.136353 7454 tablet_service.cc:1514] Processing DeleteTablet for tablet 2e3e21ae7a9d47a28cde04334c41b44b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 06:59:10 UTC) from {username='slave'} at 127.0.0.1:59556
I20260812 06:59:10.136762 7617 tablet_service.cc:1514] Processing DeleteTablet for tablet 2e3e21ae7a9d47a28cde04334c41b44b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 06:59:10 UTC) from {username='slave'} at 127.0.0.1:50238
I20260812 06:59:10.136914 7889 tablet_replica.cc:330] T 2e3e21ae7a9d47a28cde04334c41b44b P 917bef5472ff4182a74f83edc0884017: stopping tablet replica
I20260812 06:59:10.137090 7890 tablet_replica.cc:330] T 2e3e21ae7a9d47a28cde04334c41b44b P 7e187814a4a7477e8c7c6385cd56a295: stopping tablet replica
I20260812 06:59:10.137280 7889 raft_consensus.cc:2229] T 2e3e21ae7a9d47a28cde04334c41b44b P 917bef5472ff4182a74f83edc0884017 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 06:59:10.137712 7889 raft_consensus.cc:2258] T 2e3e21ae7a9d47a28cde04334c41b44b P 917bef5472ff4182a74f83edc0884017 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 06:59:10.137763 19665 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu with pid 7402
I20260812 06:59:10.139302 7891 tablet_replica.cc:330] T 2e3e21ae7a9d47a28cde04334c41b44b P e4e1d8ac0d0c43fc91e3cb5808a9f99f: stopping tablet replica
I20260812 06:59:10.139626 7891 raft_consensus.cc:2229] T 2e3e21ae7a9d47a28cde04334c41b44b P e4e1d8ac0d0c43fc91e3cb5808a9f99f [term 1 LEADER]: Raft consensus shutting down.
I20260812 06:59:10.139988 7891 raft_consensus.cc:2258] T 2e3e21ae7a9d47a28cde04334c41b44b P e4e1d8ac0d0c43fc91e3cb5808a9f99f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 06:59:10.143944 7889 ts_tablet_manager.cc:1905] T 2e3e21ae7a9d47a28cde04334c41b44b P 917bef5472ff4182a74f83edc0884017: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 06:59:10.144440 7891 ts_tablet_manager.cc:1905] T 2e3e21ae7a9d47a28cde04334c41b44b P e4e1d8ac0d0c43fc91e3cb5808a9f99f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 06:59:10.147048 7889 ts_tablet_manager.cc:1918] T 2e3e21ae7a9d47a28cde04334c41b44b P 917bef5472ff4182a74f83edc0884017: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 06:59:10.147158 7889 log.cc:1196] T 2e3e21ae7a9d47a28cde04334c41b44b P 917bef5472ff4182a74f83edc0884017: Deleting WAL directory at /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786517687384214-19665-0/minicluster-data/ts-2/wal/wals/2e3e21ae7a9d47a28cde04334c41b44b
I20260812 06:59:10.147235 7891 ts_tablet_manager.cc:1918] T 2e3e21ae7a9d47a28cde04334c41b44b P e4e1d8ac0d0c43fc91e3cb5808a9f99f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 06:59:10.147325 7891 log.cc:1196] T 2e3e21ae7a9d47a28cde04334c41b44b P e4e1d8ac0d0c43fc91e3cb5808a9f99f: Deleting WAL directory at /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786517687384214-19665-0/minicluster-data/ts-1/wal/wals/2e3e21ae7a9d47a28cde04334c41b44b
I20260812 06:59:10.147521 7889 ts_tablet_manager.cc:1939] T 2e3e21ae7a9d47a28cde04334c41b44b P 917bef5472ff4182a74f83edc0884017: Deleting consensus metadata
I20260812 06:59:10.147635 7891 ts_tablet_manager.cc:1939] T 2e3e21ae7a9d47a28cde04334c41b44b P e4e1d8ac0d0c43fc91e3cb5808a9f99f: Deleting consensus metadata
I20260812 06:59:10.148949 7316 catalog_manager.cc:4841] TS 917bef5472ff4182a74f83edc0884017 (127.19.52.67:39669): tablet 2e3e21ae7a9d47a28cde04334c41b44b (table test-table [id=761f7249f1c4489b94b1a3b1a6e81b6f]) successfully deleted
I20260812 06:59:10.149129 7313 catalog_manager.cc:4841] TS e4e1d8ac0d0c43fc91e3cb5808a9f99f (127.19.52.66:33385): tablet 2e3e21ae7a9d47a28cde04334c41b44b (table test-table [id=761f7249f1c4489b94b1a3b1a6e81b6f]) successfully deleted
W20260812 06:59:10.151305 7315 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.19.52.65:45755 (error 108)
I20260812 06:59:10.151384 19665 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu with pid 7550
I20260812 06:59:10.155658 7856 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:59:10.155177 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.65:45755 (local address 127.0.0.1:59568)
0812 06:59:10.155429 (+ 252us) negotiation.cc:107] Waiting for socket to connect
0812 06:59:10.155525 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.52.65:45755: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":138}
W20260812 06:59:10.155910 7315 catalog_manager.cc:4584] TS 7e187814a4a7477e8c7c6385cd56a295 (127.19.52.65:45755): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 2e3e21ae7a9d47a28cde04334c41b44b: Network error: Client connection negotiation failed: client connection to 127.19.52.65:45755: connect: Connection refused (error 111)
I20260812 06:59:10.162647 19665 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu with pid 7697
I20260812 06:59:10.173301 19665 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu with pid 7295
2026-08-12T06:59:10Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/1 (1248 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/2
Loading random data
Initializing database '/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786517687384214-19665-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 06:59:10 dist-test-slave-65mx krb5kdc[7895](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 06:59:10 dist-test-slave-65mx krb5kdc[7895](info): set up 2 sockets
Aug 12 06:59:10 dist-test-slave-65mx krb5kdc[7895](info): commencing operation
krb5kdc: starting...
W20260812 06:59:11.319111 19665 mini_kdc.cc:121] Time spent starting KDC: real 1.121s user 0.003s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786517687384214-19665-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786517687384214-19665-0/krb5kdc/test-user.keytab.
Aug 12 06:59:11 dist-test-slave-65mx krb5kdc[7895](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517951, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T06:59:11Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T06:59:11Z Disabled control of system clock
WARNING: no policy specified for kudu/127.19.52.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.52.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.52.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786517687384214-19665-0/krb5kdc/kudu_127.19.52.126.keytab.
Entry for principal kudu/127.19.52.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786517687384214-19665-0/krb5kdc/kudu_127.19.52.126.keytab.
I20260812 06:59:11.464422 19665 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu
/tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786517687384214-19665-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786517687384214-19665-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786517687384214-19665-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786517687384214-19665-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.52.126:34135
--webserver_interface=127.19.52.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786517687384214-19665-0/krb5kdc/kudu_127.19.52.126.keytab
--principal=kudu/127.19.52.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.52.84:41989
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.19.52.126:34135
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786517687384214-19665-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786517687384214-19665-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786517687384214-19665-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 06:59:11.585144 7910 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:59:11.585508 7910 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:59:11.585587 7910 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 06:59:11.589792 7910 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 06:59:11.589900 7910 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 06:59:11.589941 7910 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 06:59:11.589982 7910 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 06:59:11.590021 7910 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 06:59:11.595502 7910 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.52.84:41989
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786517687384214-19665-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786517687384214-19665-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.19.52.126:34135
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786517687384214-19665-0/krb5kdc/kudu_127.19.52.126.keytab
--principal=kudu/127.19.52.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.52.126:34135
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786517687384214-19665-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.19.52.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7910
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786517687384214-19665-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 06:15:54 UTC on ccycloud.quasar-rnfzce.root.comops.site
build id 4581989
I20260812 06:59:11.596740 7910 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:59:11.597931 7910 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 06:59:11.607681 7917 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:59:11.607620 7919 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:59:11.607620 7915 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:59:11.607754 7910 server_base.cc:1034] running on GCE node
I20260812 06:59:11.608736 7910 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:59:11.609956 7910 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:59:11.611261 7910 hybrid_clock.cc:648] HybridClock initialized: now 1786517951611248 us; error 31 us; skew 500 ppm
Aug 12 06:59:11 dist-test-slave-65mx krb5kdc[7895](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517951, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.52.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 06:59:11.614995 7910 init.cc:377] Logged in from keytab as kudu/127.19.52.126@KRBTEST.COM (short username kudu)
I20260812 06:59:11.616282 7910 webserver.cc:460] Webserver started at http://127.19.52.126:40301/ using document root <none> and password file <none>
I20260812 06:59:11.617137 7910 fs_manager.cc:359] Metadata directory not provided
I20260812 06:59:11.617249 7910 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:59:11.617520 7910 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 06:59:11.619851 7910 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786517687384214-19665-0/minicluster-data/master-0/data/instance:
uuid: "47b0874b048b4c79975ebd25f1fab9d7"
format_stamp: "Formatted at 2026-08-12 06:59:11 on dist-test-slave-65mx"
server_key: "59efaa44c60346559dc1b562e4f1d47a"
server_key_iv: "c7c332373f061496d832d73dcc874ef7"
server_key_version: "encryptionkey@0"
I20260812 06:59:11.620509 7910 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786517687384214-19665-0/minicluster-data/master-0/wal/instance:
uuid: "47b0874b048b4c79975ebd25f1fab9d7"
format_stamp: "Formatted at 2026-08-12 06:59:11 on dist-test-slave-65mx"
server_key: "59efaa44c60346559dc1b562e4f1d47a"
server_key_iv: "c7c332373f061496d832d73dcc874ef7"
server_key_version: "encryptionkey@0"
I20260812 06:59:11.624754 7910 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260812 06:59:11.627400 7928 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 06:59:11.628901 7910 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 06:59:11.629056 7910 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786517687384214-19665-0/minicluster-data/master-0/data,/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786517687384214-19665-0/minicluster-data/master-0/wal
uuid: "47b0874b048b4c79975ebd25f1fab9d7"
format_stamp: "Formatted at 2026-08-12 06:59:11 on dist-test-slave-65mx"
server_key: "59efaa44c60346559dc1b562e4f1d47a"
server_key_iv: "c7c332373f061496d832d73dcc874ef7"
server_key_version: "encryptionkey@0"
I20260812 06:59:11.629184 7910 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786517687384214-19665-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786517687384214-19665-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786517687384214-19665-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:59:11.647861 7910 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 06:59:11.651755 7910 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 06:59:11.652014 7910 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:59:11.661082 7910 rpc_server.cc:307] RPC server started. Bound to: 127.19.52.126:34135
I20260812 06:59:11.661121 7989 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.52.126:34135 every 8 connection(s)
I20260812 06:59:11.662207 7910 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786517687384214-19665-0/minicluster-data/master-0/data/info.pb
I20260812 06:59:11.663120 19665 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu as pid 7910
I20260812 06:59:11.663273 19665 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786517687384214-19665-0/minicluster-data/master-0/wal/instance
I20260812 06:59:11.663616 19665 external_mini_cluster.cc:1428] Setting key 73c5806eec296c7fb7eb9f48cedbfe50
I20260812 06:59:11.666095 7990 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 06:59:11.671891 7990 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 47b0874b048b4c79975ebd25f1fab9d7: Bootstrap starting.
Aug 12 06:59:11 dist-test-slave-65mx krb5kdc[7895](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517951, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.19.52.126@KRBTEST.COM
I20260812 06:59:11.674928 7990 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 47b0874b048b4c79975ebd25f1fab9d7: Neither blocks nor log segments found. Creating new log.
I20260812 06:59:11.675967 7990 log.cc:824] T 00000000000000000000000000000000 P 47b0874b048b4c79975ebd25f1fab9d7: Log is configured to *not* fsync() on all Append() calls
I20260812 06:59:11.678638 7990 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 47b0874b048b4c79975ebd25f1fab9d7: No bootstrap required, opened a new log
I20260812 06:59:11.680379 7994 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:59:11.665262 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57572 (local address 127.19.52.126:34135)
0812 06:59:11.665782 (+ 520us) server_negotiation.cc:206] Beginning negotiation
0812 06:59:11.665806 (+ 24us) server_negotiation.cc:399] Waiting for connection header
0812 06:59:11.665848 (+ 42us) server_negotiation.cc:407] Connection header received
0812 06:59:11.666676 (+ 828us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:59:11.666697 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:59:11.667061 (+ 364us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:59:11.667437 (+ 376us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:59:11.668364 (+ 927us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:59:11.669586 (+ 1222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:59:11.670581 (+ 995us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:59:11.671093 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:59:11.671275 (+ 182us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:59:11.674752 (+ 3477us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:59:11.674762 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:59:11.674783 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:59:11.674828 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:59:11.676951 (+ 2123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:59:11.677462 (+ 511us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:59:11.677468 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:59:11.677474 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:59:11.677625 (+ 151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:59:11.677904 (+ 279us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:59:11.677908 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:59:11.677910 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:59:11.678439 (+ 529us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:59:11.678667 (+ 228us) server_negotiation.cc:1035] Waiting for connection context
0812 06:59:11.679039 (+ 372us) server_negotiation.cc:299] Negotiation successful
0812 06:59:11.679369 (+ 330us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":375,"thread_start_us":184,"threads_started":1}
I20260812 06:59:11.683256 7990 raft_consensus.cc:348] T 00000000000000000000000000000000 P 47b0874b048b4c79975ebd25f1fab9d7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "47b0874b048b4c79975ebd25f1fab9d7" member_type: VOTER last_known_addr { host: "127.19.52.126" port: 34135 } }
I20260812 06:59:11.683630 7990 raft_consensus.cc:374] T 00000000000000000000000000000000 P 47b0874b048b4c79975ebd25f1fab9d7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 06:59:11.683701 7990 raft_consensus.cc:729] T 00000000000000000000000000000000 P 47b0874b048b4c79975ebd25f1fab9d7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 47b0874b048b4c79975ebd25f1fab9d7, State: Initialized, Role: FOLLOWER
I20260812 06:59:11.684108 7990 consensus_queue.cc:260] T 00000000000000000000000000000000 P 47b0874b048b4c79975ebd25f1fab9d7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "47b0874b048b4c79975ebd25f1fab9d7" member_type: VOTER last_known_addr { host: "127.19.52.126" port: 34135 } }
I20260812 06:59:11.684237 7990 raft_consensus.cc:388] T 00000000000000000000000000000000 P 47b0874b048b4c79975ebd25f1fab9d7 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 06:59:11.684281 7990 raft_consensus.cc:482] T 00000000000000000000000000000000 P 47b0874b048b4c79975ebd25f1fab9d7 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 06:59:11.684353 7990 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 47b0874b048b4c79975ebd25f1fab9d7 [term 0 FOLLOWER]: Advancing to term 1
I20260812 06:59:11.685418 7990 raft_consensus.cc:504] T 00000000000000000000000000000000 P 47b0874b048b4c79975ebd25f1fab9d7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "47b0874b048b4c79975ebd25f1fab9d7" member_type: VOTER last_known_addr { host: "127.19.52.126" port: 34135 } }
I20260812 06:59:11.685725 7990 leader_election.cc:302] T 00000000000000000000000000000000 P 47b0874b048b4c79975ebd25f1fab9d7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 47b0874b048b4c79975ebd25f1fab9d7; no voters:
I20260812 06:59:11.686000 7990 leader_election.cc:288] T 00000000000000000000000000000000 P 47b0874b048b4c79975ebd25f1fab9d7 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 06:59:11.686209 7996 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 47b0874b048b4c79975ebd25f1fab9d7 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 06:59:11.686550 7996 raft_consensus.cc:686] T 00000000000000000000000000000000 P 47b0874b048b4c79975ebd25f1fab9d7 [term 1 LEADER]: Becoming Leader. State: Replica: 47b0874b048b4c79975ebd25f1fab9d7, State: Running, Role: LEADER
I20260812 06:59:11.686878 7996 consensus_queue.cc:237] T 00000000000000000000000000000000 P 47b0874b048b4c79975ebd25f1fab9d7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "47b0874b048b4c79975ebd25f1fab9d7" member_type: VOTER last_known_addr { host: "127.19.52.126" port: 34135 } }
I20260812 06:59:11.687418 7990 sys_catalog.cc:564] T 00000000000000000000000000000000 P 47b0874b048b4c79975ebd25f1fab9d7 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 06:59:11.688683 7997 sys_catalog.cc:455] T 00000000000000000000000000000000 P 47b0874b048b4c79975ebd25f1fab9d7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "47b0874b048b4c79975ebd25f1fab9d7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "47b0874b048b4c79975ebd25f1fab9d7" member_type: VOTER last_known_addr { host: "127.19.52.126" port: 34135 } } }
I20260812 06:59:11.688957 7998 sys_catalog.cc:455] T 00000000000000000000000000000000 P 47b0874b048b4c79975ebd25f1fab9d7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 47b0874b048b4c79975ebd25f1fab9d7. Latest consensus state: current_term: 1 leader_uuid: "47b0874b048b4c79975ebd25f1fab9d7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "47b0874b048b4c79975ebd25f1fab9d7" member_type: VOTER last_known_addr { host: "127.19.52.126" port: 34135 } } }
I20260812 06:59:11.689090 7997 sys_catalog.cc:458] T 00000000000000000000000000000000 P 47b0874b048b4c79975ebd25f1fab9d7 [sys.catalog]: This master's current role is: LEADER
I20260812 06:59:11.689465 7998 sys_catalog.cc:458] T 00000000000000000000000000000000 P 47b0874b048b4c79975ebd25f1fab9d7 [sys.catalog]: This master's current role is: LEADER
W20260812 06:59:11.692412 8013 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 47b0874b048b4c79975ebd25f1fab9d7: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 06:59:11.692521 8013 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 06:59:11.692641 8012 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 06:59:11.693699 8012 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 06:59:11.699930 8012 catalog_manager.cc:1347] Generated new cluster ID: 32898d33e78948d2a69e19a3fd2a653a
I20260812 06:59:11.700038 8012 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 06:59:11.720132 8012 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 06:59:11.722083 8012 catalog_manager.cc:1504] Loading token signing keys...
I20260812 06:59:11.731823 8012 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 47b0874b048b4c79975ebd25f1fab9d7: Generated new TSK 0
I20260812 06:59:11.733131 8012 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.19.52.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.52.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.52.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786517687384214-19665-0/krb5kdc/kudu_127.19.52.65.keytab.
Entry for principal kudu/127.19.52.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786517687384214-19665-0/krb5kdc/kudu_127.19.52.65.keytab.
I20260812 06:59:11.769718 19665 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu
/tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786517687384214-19665-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786517687384214-19665-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786517687384214-19665-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786517687384214-19665-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.52.65:0
--local_ip_for_outbound_sockets=127.19.52.65
--webserver_interface=127.19.52.65
--webserver_port=0
--tserver_master_addrs=127.19.52.126:34135
--keytab_file=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786517687384214-19665-0/krb5kdc/kudu_127.19.52.65.keytab
--principal=kudu/127.19.52.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.52.84:41989
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786517687384214-19665-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786517687384214-19665-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786517687384214-19665-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 06:59:11.889696 8020 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:59:11.890000 8020 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:59:11.890048 8020 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 06:59:11.894361 8020 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 06:59:11.894433 8020 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 06:59:11.894522 8020 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.52.65
I20260812 06:59:11.900113 8020 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.52.84:41989
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786517687384214-19665-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786517687384214-19665-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786517687384214-19665-0/krb5kdc/kudu_127.19.52.65.keytab
--principal=kudu/127.19.52.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.52.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786517687384214-19665-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.19.52.65
--webserver_port=0
--tserver_master_addrs=127.19.52.126:34135
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8020
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.52.65
--log_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786517687384214-19665-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 06:15:54 UTC on ccycloud.quasar-rnfzce.root.comops.site
build id 4581989
I20260812 06:59:11.901283 8020 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:59:11.902338 8020 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 06:59:11.915012 8025 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:59:11.915012 8027 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:59:11.915040 8030 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:59:11.915000 8020 server_base.cc:1034] running on GCE node
I20260812 06:59:11.915724 8020 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:59:11.916708 8020 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:59:11.924642 8020 hybrid_clock.cc:648] HybridClock initialized: now 1786517951924579 us; error 61 us; skew 500 ppm
Aug 12 06:59:11 dist-test-slave-65mx krb5kdc[7895](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517951, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.52.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 06:59:11.929189 8020 init.cc:377] Logged in from keytab as kudu/127.19.52.65@KRBTEST.COM (short username kudu)
I20260812 06:59:11.930810 8020 webserver.cc:460] Webserver started at http://127.19.52.65:32949/ using document root <none> and password file <none>
I20260812 06:59:11.931555 8020 fs_manager.cc:359] Metadata directory not provided
I20260812 06:59:11.931653 8020 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:59:11.931900 8020 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 06:59:11.935426 8020 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786517687384214-19665-0/minicluster-data/ts-0/data/instance:
uuid: "9b9a1d0e5ee647aa95c0c14941ce99ff"
format_stamp: "Formatted at 2026-08-12 06:59:11 on dist-test-slave-65mx"
server_key: "16b1f047d529a0b4da786476f9272238"
server_key_iv: "0c14f115ac9b24efa672ec4086e14539"
server_key_version: "encryptionkey@0"
I20260812 06:59:11.936342 8020 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786517687384214-19665-0/minicluster-data/ts-0/wal/instance:
uuid: "9b9a1d0e5ee647aa95c0c14941ce99ff"
format_stamp: "Formatted at 2026-08-12 06:59:11 on dist-test-slave-65mx"
server_key: "16b1f047d529a0b4da786476f9272238"
server_key_iv: "0c14f115ac9b24efa672ec4086e14539"
server_key_version: "encryptionkey@0"
I20260812 06:59:11.942936 8020 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.004s sys 0.000s
I20260812 06:59:11.946552 8038 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 06:59:11.948360 8020 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260812 06:59:11.948585 8020 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786517687384214-19665-0/minicluster-data/ts-0/data,/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786517687384214-19665-0/minicluster-data/ts-0/wal
uuid: "9b9a1d0e5ee647aa95c0c14941ce99ff"
format_stamp: "Formatted at 2026-08-12 06:59:11 on dist-test-slave-65mx"
server_key: "16b1f047d529a0b4da786476f9272238"
server_key_iv: "0c14f115ac9b24efa672ec4086e14539"
server_key_version: "encryptionkey@0"
I20260812 06:59:11.948762 8020 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786517687384214-19665-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786517687384214-19665-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786517687384214-19665-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:59:11.976069 8020 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 06:59:11.979848 8020 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 06:59:11.980074 8020 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:59:11.980774 8020 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 06:59:11.981643 8020 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 06:59:11.981717 8020 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 06:59:11.981789 8020 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 06:59:11.981841 8020 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 06:59:11.993377 8020 rpc_server.cc:307] RPC server started. Bound to: 127.19.52.65:38133
I20260812 06:59:11.993476 8175 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.52.65:38133 every 8 connection(s)
I20260812 06:59:11.994382 8020 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786517687384214-19665-0/minicluster-data/ts-0/data/info.pb
I20260812 06:59:11.998565 19665 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu as pid 8020
I20260812 06:59:11.998714 19665 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786517687384214-19665-0/minicluster-data/ts-0/wal/instance
I20260812 06:59:11.999018 19665 external_mini_cluster.cc:1428] Setting key 3c9bda6dff038a9ef0524e5cd30d0812
Aug 12 06:59:12 dist-test-slave-65mx krb5kdc[7895](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517951, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.52.65@KRBTEST.COM for kudu/127.19.52.126@KRBTEST.COM
I20260812 06:59:12.010032 7994 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:59:11.996296 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.52.65:45165 (local address 127.19.52.126:34135)
0812 06:59:11.996450 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0812 06:59:11.996454 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 06:59:11.997467 (+ 1013us) server_negotiation.cc:407] Connection header received
0812 06:59:11.998948 (+ 1481us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:59:11.998951 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:59:11.999012 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:59:11.999087 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:59:12.000286 (+ 1199us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:59:12.000896 (+ 610us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:59:12.002087 (+ 1191us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:59:12.002386 (+ 299us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:59:12.002638 (+ 252us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:59:12.005582 (+ 2944us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:59:12.005595 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:59:12.005599 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:59:12.005641 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:59:12.007578 (+ 1937us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:59:12.008140 (+ 562us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:59:12.008145 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:59:12.008149 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:59:12.008249 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:59:12.008725 (+ 476us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:59:12.008728 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:59:12.008730 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:59:12.008995 (+ 265us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:59:12.009108 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0812 06:59:12.009672 (+ 564us) server_negotiation.cc:299] Negotiation successful
0812 06:59:12.009823 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
WARNING: no policy specified for kudu/127.19.52.66@KRBTEST.COM; defaulting to no policy
I20260812 06:59:12.010848 8178 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:59:11.996783 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.126:34135 (local address 127.19.52.65:45165)
0812 06:59:11.997282 (+ 499us) negotiation.cc:107] Waiting for socket to connect
0812 06:59:11.997330 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0812 06:59:11.998701 (+ 1371us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:59:11.999260 (+ 559us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:59:11.999267 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:59:11.999628 (+ 361us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:59:12.000128 (+ 500us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:59:12.000142 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:59:12.001030 (+ 888us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:59:12.001033 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:59:12.001933 (+ 900us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:59:12.001943 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:59:12.002664 (+ 721us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:59:12.002667 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:59:12.002823 (+ 156us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:59:12.003482 (+ 659us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 06:59:12.003499 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:59:12.005372 (+ 1873us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 06:59:12.007739 (+ 2367us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:59:12.007744 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:59:12.007758 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:59:12.008029 (+ 271us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:59:12.008353 (+ 324us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:59:12.008355 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:59:12.008356 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:59:12.008503 (+ 147us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:59:12.009126 (+ 623us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 06:59:12.009132 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 06:59:12.009388 (+ 256us) client_negotiation.cc:769] Sending connection context
0812 06:59:12.009567 (+ 179us) client_negotiation.cc:240] Negotiation successful
0812 06:59:12.009862 (+ 295us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":297,"thread_start_us":133,"threads_started":1}
I20260812 06:59:12.012197 8176 heartbeater.cc:347] Connected to a master server at 127.19.52.126:34135
I20260812 06:59:12.012450 8176 heartbeater.cc:464] Registering TS with master...
I20260812 06:59:12.013019 8176 heartbeater.cc:511] Master 127.19.52.126:34135 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.52.66@KRBTEST.COM" created.
I20260812 06:59:12.014726 7947 ts_manager.cc:194] Registered new tserver with Master: 9b9a1d0e5ee647aa95c0c14941ce99ff (127.19.52.65:38133)
I20260812 06:59:12.016376 7947 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.52.65@KRBTEST.COM'} at 127.19.52.65:45165
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.52.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786517687384214-19665-0/krb5kdc/kudu_127.19.52.66.keytab.
Entry for principal kudu/127.19.52.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786517687384214-19665-0/krb5kdc/kudu_127.19.52.66.keytab.
I20260812 06:59:12.036371 19665 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu
/tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786517687384214-19665-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786517687384214-19665-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786517687384214-19665-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786517687384214-19665-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.52.66:0
--local_ip_for_outbound_sockets=127.19.52.66
--webserver_interface=127.19.52.66
--webserver_port=0
--tserver_master_addrs=127.19.52.126:34135
--keytab_file=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786517687384214-19665-0/krb5kdc/kudu_127.19.52.66.keytab
--principal=kudu/127.19.52.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.52.84:41989
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786517687384214-19665-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786517687384214-19665-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786517687384214-19665-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 06:59:12.155309 8181 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:59:12.155611 8181 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:59:12.155656 8181 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 06:59:12.160269 8181 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 06:59:12.160365 8181 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 06:59:12.160496 8181 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.52.66
I20260812 06:59:12.166278 8181 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.52.84:41989
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786517687384214-19665-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786517687384214-19665-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786517687384214-19665-0/krb5kdc/kudu_127.19.52.66.keytab
--principal=kudu/127.19.52.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.52.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786517687384214-19665-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.19.52.66
--webserver_port=0
--tserver_master_addrs=127.19.52.126:34135
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8181
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.52.66
--log_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786517687384214-19665-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 06:15:54 UTC on ccycloud.quasar-rnfzce.root.comops.site
build id 4581989
I20260812 06:59:12.167469 8181 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:59:12.168531 8181 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 06:59:12.178891 8191 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:59:12.178953 8181 server_base.cc:1034] running on GCE node
W20260812 06:59:12.178869 8188 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:59:12.178891 8189 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:59:12.179577 8181 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:59:12.180284 8181 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:59:12.181454 8181 hybrid_clock.cc:648] HybridClock initialized: now 1786517952181422 us; error 42 us; skew 500 ppm
Aug 12 06:59:12 dist-test-slave-65mx krb5kdc[7895](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517952, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.52.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 06:59:12.185019 8181 init.cc:377] Logged in from keytab as kudu/127.19.52.66@KRBTEST.COM (short username kudu)
I20260812 06:59:12.186050 8181 webserver.cc:460] Webserver started at http://127.19.52.66:32815/ using document root <none> and password file <none>
I20260812 06:59:12.186687 8181 fs_manager.cc:359] Metadata directory not provided
I20260812 06:59:12.186755 8181 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:59:12.187047 8181 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 06:59:12.189613 8181 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786517687384214-19665-0/minicluster-data/ts-1/data/instance:
uuid: "1ccf14d1ef9c4e4eb9a6225273c4f345"
format_stamp: "Formatted at 2026-08-12 06:59:12 on dist-test-slave-65mx"
server_key: "bf2c81cdc71888e1201ada2287102466"
server_key_iv: "7d8058da682b618124223e866739047e"
server_key_version: "encryptionkey@0"
I20260812 06:59:12.190325 8181 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786517687384214-19665-0/minicluster-data/ts-1/wal/instance:
uuid: "1ccf14d1ef9c4e4eb9a6225273c4f345"
format_stamp: "Formatted at 2026-08-12 06:59:12 on dist-test-slave-65mx"
server_key: "bf2c81cdc71888e1201ada2287102466"
server_key_iv: "7d8058da682b618124223e866739047e"
server_key_version: "encryptionkey@0"
I20260812 06:59:12.194427 8181 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260812 06:59:12.197083 8198 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 06:59:12.198586 8181 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 06:59:12.198771 8181 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786517687384214-19665-0/minicluster-data/ts-1/data,/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786517687384214-19665-0/minicluster-data/ts-1/wal
uuid: "1ccf14d1ef9c4e4eb9a6225273c4f345"
format_stamp: "Formatted at 2026-08-12 06:59:12 on dist-test-slave-65mx"
server_key: "bf2c81cdc71888e1201ada2287102466"
server_key_iv: "7d8058da682b618124223e866739047e"
server_key_version: "encryptionkey@0"
I20260812 06:59:12.198905 8181 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786517687384214-19665-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786517687384214-19665-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786517687384214-19665-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:59:12.215342 8181 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 06:59:12.219340 8181 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 06:59:12.219575 8181 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:59:12.220238 8181 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 06:59:12.221203 8181 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 06:59:12.221277 8181 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 06:59:12.221356 8181 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 06:59:12.221408 8181 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 06:59:12.231669 8181 rpc_server.cc:307] RPC server started. Bound to: 127.19.52.66:42999
I20260812 06:59:12.231719 8328 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.52.66:42999 every 8 connection(s)
I20260812 06:59:12.232682 8181 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786517687384214-19665-0/minicluster-data/ts-1/data/info.pb
I20260812 06:59:12.235173 19665 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu as pid 8181
I20260812 06:59:12.235338 19665 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786517687384214-19665-0/minicluster-data/ts-1/wal/instance
I20260812 06:59:12.235666 19665 external_mini_cluster.cc:1428] Setting key 9506abe7ed32a2cb0a30f008ad3a0e4c
Aug 12 06:59:12 dist-test-slave-65mx krb5kdc[7895](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517952, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.52.66@KRBTEST.COM for kudu/127.19.52.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.19.52.67@KRBTEST.COM; defaulting to no policy
I20260812 06:59:12.249322 7994 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:59:12.234628 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.52.66:39269 (local address 127.19.52.126:34135)
0812 06:59:12.234768 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0812 06:59:12.234773 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 06:59:12.235777 (+ 1004us) server_negotiation.cc:407] Connection header received
0812 06:59:12.237480 (+ 1703us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:59:12.237484 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:59:12.237533 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:59:12.237610 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:59:12.239323 (+ 1713us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:59:12.239911 (+ 588us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:59:12.241719 (+ 1808us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:59:12.241935 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:59:12.242029 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:59:12.245124 (+ 3095us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:59:12.245131 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:59:12.245137 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:59:12.245171 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:59:12.246905 (+ 1734us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:59:12.247522 (+ 617us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:59:12.247526 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:59:12.247528 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:59:12.247621 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:59:12.248087 (+ 466us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:59:12.248091 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:59:12.248093 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:59:12.248300 (+ 207us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:59:12.248384 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0812 06:59:12.249005 (+ 621us) server_negotiation.cc:299] Negotiation successful
0812 06:59:12.249147 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260812 06:59:12.250324 8331 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:59:12.234963 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.126:34135 (local address 127.19.52.66:39269)
0812 06:59:12.235565 (+ 602us) negotiation.cc:107] Waiting for socket to connect
0812 06:59:12.235631 (+ 66us) client_negotiation.cc:174] Beginning negotiation
0812 06:59:12.237201 (+ 1570us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:59:12.237824 (+ 623us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:59:12.237833 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:59:12.238398 (+ 565us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:59:12.239131 (+ 733us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:59:12.239153 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:59:12.240072 (+ 919us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:59:12.240077 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:59:12.241577 (+ 1500us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:59:12.241592 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:59:12.242115 (+ 523us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:59:12.242122 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:59:12.242370 (+ 248us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:59:12.243058 (+ 688us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 06:59:12.243083 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:59:12.244921 (+ 1838us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 06:59:12.247073 (+ 2152us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:59:12.247078 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:59:12.247098 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:59:12.247383 (+ 285us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:59:12.247749 (+ 366us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:59:12.247755 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:59:12.247756 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:59:12.247929 (+ 173us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:59:12.248402 (+ 473us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 06:59:12.248408 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 06:59:12.248736 (+ 328us) client_negotiation.cc:769] Sending connection context
0812 06:59:12.248939 (+ 203us) client_negotiation.cc:240] Negotiation successful
0812 06:59:12.249365 (+ 426us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":297,"thread_start_us":121,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.52.67@KRBTEST.COM" created.
I20260812 06:59:12.251843 8329 heartbeater.cc:347] Connected to a master server at 127.19.52.126:34135
I20260812 06:59:12.252130 8329 heartbeater.cc:464] Registering TS with master...
I20260812 06:59:12.252755 8329 heartbeater.cc:511] Master 127.19.52.126:34135 requested a full tablet report, sending...
I20260812 06:59:12.253885 7947 ts_manager.cc:194] Registered new tserver with Master: 1ccf14d1ef9c4e4eb9a6225273c4f345 (127.19.52.66:42999)
I20260812 06:59:12.254508 7947 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.52.66@KRBTEST.COM'} at 127.19.52.66:39269
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.52.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786517687384214-19665-0/krb5kdc/kudu_127.19.52.67.keytab.
Entry for principal kudu/127.19.52.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786517687384214-19665-0/krb5kdc/kudu_127.19.52.67.keytab.
I20260812 06:59:12.271631 19665 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu
/tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786517687384214-19665-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786517687384214-19665-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786517687384214-19665-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786517687384214-19665-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.52.67:0
--local_ip_for_outbound_sockets=127.19.52.67
--webserver_interface=127.19.52.67
--webserver_port=0
--tserver_master_addrs=127.19.52.126:34135
--keytab_file=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786517687384214-19665-0/krb5kdc/kudu_127.19.52.67.keytab
--principal=kudu/127.19.52.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.52.84:41989
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786517687384214-19665-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786517687384214-19665-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786517687384214-19665-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 06:59:12.389518 8335 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:59:12.389832 8335 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:59:12.389914 8335 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 06:59:12.394110 8335 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 06:59:12.394205 8335 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 06:59:12.394296 8335 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.52.67
I20260812 06:59:12.399848 8335 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.52.84:41989
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786517687384214-19665-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786517687384214-19665-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786517687384214-19665-0/krb5kdc/kudu_127.19.52.67.keytab
--principal=kudu/127.19.52.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.52.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786517687384214-19665-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.19.52.67
--webserver_port=0
--tserver_master_addrs=127.19.52.126:34135
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8335
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.52.67
--log_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786517687384214-19665-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 06:15:54 UTC on ccycloud.quasar-rnfzce.root.comops.site
build id 4581989
I20260812 06:59:12.401082 8335 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:59:12.402119 8335 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260812 06:59:12.412497 8335 server_base.cc:1034] running on GCE node
W20260812 06:59:12.412452 8343 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:59:12.412442 8340 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:59:12.412442 8341 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:59:12.413264 8335 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:59:12.414012 8335 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:59:12.415215 8335 hybrid_clock.cc:648] HybridClock initialized: now 1786517952415196 us; error 32 us; skew 500 ppm
Aug 12 06:59:12 dist-test-slave-65mx krb5kdc[7895](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517952, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.52.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 06:59:12.418395 8335 init.cc:377] Logged in from keytab as kudu/127.19.52.67@KRBTEST.COM (short username kudu)
I20260812 06:59:12.419600 8335 webserver.cc:460] Webserver started at http://127.19.52.67:35255/ using document root <none> and password file <none>
I20260812 06:59:12.420228 8335 fs_manager.cc:359] Metadata directory not provided
I20260812 06:59:12.420331 8335 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:59:12.420646 8335 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 06:59:12.422838 8335 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786517687384214-19665-0/minicluster-data/ts-2/data/instance:
uuid: "72bd459b398e4bdb9538c7ac0d7dccb2"
format_stamp: "Formatted at 2026-08-12 06:59:12 on dist-test-slave-65mx"
server_key: "61093096a9ad583340d5cb2a06d5da0a"
server_key_iv: "b1dea06d8f4a05f71f08f2bc3ee59dec"
server_key_version: "encryptionkey@0"
I20260812 06:59:12.423452 8335 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786517687384214-19665-0/minicluster-data/ts-2/wal/instance:
uuid: "72bd459b398e4bdb9538c7ac0d7dccb2"
format_stamp: "Formatted at 2026-08-12 06:59:12 on dist-test-slave-65mx"
server_key: "61093096a9ad583340d5cb2a06d5da0a"
server_key_iv: "b1dea06d8f4a05f71f08f2bc3ee59dec"
server_key_version: "encryptionkey@0"
I20260812 06:59:12.427431 8335 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260812 06:59:12.430258 8352 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 06:59:12.431648 8335 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260812 06:59:12.431809 8335 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786517687384214-19665-0/minicluster-data/ts-2/data,/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786517687384214-19665-0/minicluster-data/ts-2/wal
uuid: "72bd459b398e4bdb9538c7ac0d7dccb2"
format_stamp: "Formatted at 2026-08-12 06:59:12 on dist-test-slave-65mx"
server_key: "61093096a9ad583340d5cb2a06d5da0a"
server_key_iv: "b1dea06d8f4a05f71f08f2bc3ee59dec"
server_key_version: "encryptionkey@0"
I20260812 06:59:12.431941 8335 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786517687384214-19665-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786517687384214-19665-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786517687384214-19665-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:59:12.455389 8335 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 06:59:12.459013 8335 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 06:59:12.459280 8335 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:59:12.459929 8335 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 06:59:12.460886 8335 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 06:59:12.460963 8335 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 06:59:12.461050 8335 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 06:59:12.461102 8335 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 06:59:12.471053 8335 rpc_server.cc:307] RPC server started. Bound to: 127.19.52.67:33421
I20260812 06:59:12.471091 8475 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.52.67:33421 every 8 connection(s)
I20260812 06:59:12.472071 8335 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786517687384214-19665-0/minicluster-data/ts-2/data/info.pb
I20260812 06:59:12.480330 19665 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu as pid 8335
I20260812 06:59:12.480605 19665 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786517687384214-19665-0/minicluster-data/ts-2/wal/instance
I20260812 06:59:12.481017 19665 external_mini_cluster.cc:1428] Setting key 4b231abc838772196affe1002cfff020
Aug 12 06:59:12 dist-test-slave-65mx krb5kdc[7895](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517952, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.52.67@KRBTEST.COM for kudu/127.19.52.126@KRBTEST.COM
I20260812 06:59:12.488816 7994 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:59:12.474216 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.52.67:33875 (local address 127.19.52.126:34135)
0812 06:59:12.474382 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0812 06:59:12.474387 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 06:59:12.475401 (+ 1014us) server_negotiation.cc:407] Connection header received
0812 06:59:12.476833 (+ 1432us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:59:12.476836 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:59:12.476897 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:59:12.476996 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:59:12.478602 (+ 1606us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:59:12.479183 (+ 581us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:59:12.480902 (+ 1719us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:59:12.481228 (+ 326us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:59:12.481324 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:59:12.484681 (+ 3357us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:59:12.484684 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:59:12.484686 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:59:12.484714 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:59:12.486297 (+ 1583us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:59:12.486890 (+ 593us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:59:12.486893 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:59:12.486894 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:59:12.486977 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:59:12.487379 (+ 402us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:59:12.487382 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:59:12.487384 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:59:12.487654 (+ 270us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:59:12.487787 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0812 06:59:12.488448 (+ 661us) server_negotiation.cc:299] Negotiation successful
0812 06:59:12.488643 (+ 195us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260812 06:59:12.489722 8478 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:59:12.474708 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.126:34135 (local address 127.19.52.67:33875)
0812 06:59:12.475230 (+ 522us) negotiation.cc:107] Waiting for socket to connect
0812 06:59:12.475277 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0812 06:59:12.476613 (+ 1336us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:59:12.477346 (+ 733us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:59:12.477353 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:59:12.477762 (+ 409us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:59:12.478263 (+ 501us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:59:12.478277 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:59:12.479347 (+ 1070us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:59:12.479350 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:59:12.480742 (+ 1392us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:59:12.480761 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:59:12.481407 (+ 646us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:59:12.481412 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:59:12.481574 (+ 162us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:59:12.482314 (+ 740us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 06:59:12.482331 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:59:12.484334 (+ 2003us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 06:59:12.486440 (+ 2106us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:59:12.486446 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:59:12.486464 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:59:12.486764 (+ 300us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:59:12.487106 (+ 342us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:59:12.487109 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:59:12.487110 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:59:12.487259 (+ 149us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:59:12.487805 (+ 546us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 06:59:12.487839 (+ 34us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 06:59:12.488189 (+ 350us) client_negotiation.cc:769] Sending connection context
0812 06:59:12.488380 (+ 191us) client_negotiation.cc:240] Negotiation successful
0812 06:59:12.488792 (+ 412us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":330,"thread_start_us":130,"threads_started":1}
I20260812 06:59:12.491060 8476 heartbeater.cc:347] Connected to a master server at 127.19.52.126:34135
I20260812 06:59:12.491350 8476 heartbeater.cc:464] Registering TS with master...
I20260812 06:59:12.491909 8476 heartbeater.cc:511] Master 127.19.52.126:34135 requested a full tablet report, sending...
I20260812 06:59:12.493129 7947 ts_manager.cc:194] Registered new tserver with Master: 72bd459b398e4bdb9538c7ac0d7dccb2 (127.19.52.67:33421)
I20260812 06:59:12.493714 7947 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.52.67@KRBTEST.COM'} at 127.19.52.67:33875
I20260812 06:59:12.496013 19665 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 06:59:12.507270 7994 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:59:12.498456 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57588 (local address 127.19.52.126:34135)
0812 06:59:12.498623 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0812 06:59:12.498629 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 06:59:12.498749 (+ 120us) server_negotiation.cc:407] Connection header received
0812 06:59:12.498917 (+ 168us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:59:12.498924 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:59:12.498966 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:59:12.499052 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:59:12.499810 (+ 758us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:59:12.500330 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:59:12.501270 (+ 940us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:59:12.501495 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:59:12.501612 (+ 117us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:59:12.503435 (+ 1823us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:59:12.503439 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:59:12.503442 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:59:12.503478 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:59:12.505245 (+ 1767us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:59:12.505834 (+ 589us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:59:12.505837 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:59:12.505839 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:59:12.505930 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:59:12.506285 (+ 355us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:59:12.506288 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:59:12.506290 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:59:12.506530 (+ 240us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:59:12.506696 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0812 06:59:12.506986 (+ 290us) server_negotiation.cc:299] Negotiation successful
0812 06:59:12.507089 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":89}
I20260812 06:59:12.510882 7947 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57588:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 06:59:12.514374 7947 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 06:59:12.529922 8494 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:59:12.523667 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.65:38133 (local address 127.0.0.1:44362)
0812 06:59:12.524395 (+ 728us) negotiation.cc:107] Waiting for socket to connect
0812 06:59:12.524459 (+ 64us) client_negotiation.cc:174] Beginning negotiation
0812 06:59:12.524637 (+ 178us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:59:12.525508 (+ 871us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:59:12.525514 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:59:12.525547 (+ 33us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 06:59:12.525854 (+ 307us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:59:12.525861 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:59:12.527365 (+ 1504us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:59:12.527368 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:59:12.528344 (+ 976us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:59:12.528353 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:59:12.529318 (+ 965us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:59:12.529322 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:59:12.529447 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:59:12.529487 (+ 40us) client_negotiation.cc:769] Sending connection context
0812 06:59:12.529604 (+ 117us) client_negotiation.cc:240] Negotiation successful
0812 06:59:12.529740 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":551,"mutex_wait_us":176,"spinlock_wait_cycles":9600,"thread_start_us":81,"threads_started":1}
I20260812 06:59:12.529946 8490 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:59:12.523865 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.67:33421 (local address 127.0.0.1:53298)
0812 06:59:12.524394 (+ 529us) negotiation.cc:107] Waiting for socket to connect
0812 06:59:12.524459 (+ 65us) client_negotiation.cc:174] Beginning negotiation
0812 06:59:12.524637 (+ 178us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:59:12.525513 (+ 876us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:59:12.525515 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:59:12.525547 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 06:59:12.525732 (+ 185us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:59:12.525741 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:59:12.527327 (+ 1586us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:59:12.527330 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:59:12.528518 (+ 1188us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:59:12.528528 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:59:12.529589 (+ 1061us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:59:12.529596 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:59:12.529696 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:59:12.529717 (+ 21us) client_negotiation.cc:769] Sending connection context
0812 06:59:12.529790 (+ 73us) client_negotiation.cc:240] Negotiation successful
0812 06:59:12.529846 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":483,"mutex_wait_us":52,"spinlock_wait_cycles":9856,"thread_start_us":115,"threads_started":1}
I20260812 06:59:12.530244 8489 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:59:12.523674 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.66:42999 (local address 127.0.0.1:59692)
0812 06:59:12.524436 (+ 762us) negotiation.cc:107] Waiting for socket to connect
0812 06:59:12.524473 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0812 06:59:12.524637 (+ 164us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:59:12.525669 (+ 1032us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:59:12.525673 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:59:12.525688 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 06:59:12.525809 (+ 121us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:59:12.525814 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:59:12.526823 (+ 1009us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:59:12.526829 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:59:12.528416 (+ 1587us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:59:12.528424 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:59:12.529789 (+ 1365us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:59:12.529792 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:59:12.529890 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:59:12.529913 (+ 23us) client_negotiation.cc:769] Sending connection context
0812 06:59:12.529998 (+ 85us) client_negotiation.cc:240] Negotiation successful
0812 06:59:12.530072 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":693,"mutex_wait_us":569,"spinlock_wait_cycles":22528,"thread_start_us":106,"threads_started":1}
I20260812 06:59:12.530722 8495 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:59:12.524595 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53298 (local address 127.19.52.67:33421)
0812 06:59:12.525056 (+ 461us) server_negotiation.cc:206] Beginning negotiation
0812 06:59:12.525068 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0812 06:59:12.525087 (+ 19us) server_negotiation.cc:407] Connection header received
0812 06:59:12.525150 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:59:12.525154 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:59:12.525326 (+ 172us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:59:12.525896 (+ 570us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 06:59:12.526064 (+ 168us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:59:12.527190 (+ 1126us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:59:12.528673 (+ 1483us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:59:12.529491 (+ 818us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:59:12.529589 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:59:12.529718 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0812 06:59:12.530435 (+ 717us) server_negotiation.cc:299] Negotiation successful
0812 06:59:12.530559 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":381,"thread_start_us":154,"threads_started":1}
I20260812 06:59:12.530759 8488 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:59:12.523666 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44362 (local address 127.19.52.65:38133)
0812 06:59:12.524960 (+ 1294us) server_negotiation.cc:206] Beginning negotiation
0812 06:59:12.524969 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 06:59:12.524995 (+ 26us) server_negotiation.cc:407] Connection header received
0812 06:59:12.525132 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:59:12.525138 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:59:12.525326 (+ 188us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:59:12.525620 (+ 294us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 06:59:12.525980 (+ 360us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:59:12.527190 (+ 1210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:59:12.528527 (+ 1337us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:59:12.529182 (+ 655us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:59:12.529318 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:59:12.529484 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0812 06:59:12.529815 (+ 331us) server_negotiation.cc:299] Negotiation successful
0812 06:59:12.529932 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1166,"thread_start_us":123,"threads_started":1}
I20260812 06:59:12.530791 8492 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:59:12.523861 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59692 (local address 127.19.52.66:42999)
0812 06:59:12.524959 (+ 1098us) server_negotiation.cc:206] Beginning negotiation
0812 06:59:12.524982 (+ 23us) server_negotiation.cc:399] Waiting for connection header
0812 06:59:12.525007 (+ 25us) server_negotiation.cc:407] Connection header received
0812 06:59:12.525091 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:59:12.525098 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:59:12.525312 (+ 214us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:59:12.525471 (+ 159us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 06:59:12.525931 (+ 460us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:59:12.526686 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:59:12.528585 (+ 1899us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:59:12.529471 (+ 886us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:59:12.529589 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:59:12.529718 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0812 06:59:12.530394 (+ 676us) server_negotiation.cc:299] Negotiation successful
0812 06:59:12.530631 (+ 237us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":980,"thread_start_us":130,"threads_started":1}
I20260812 06:59:12.533826 8402 tablet_service.cc:1467] Processing CreateTablet for tablet 8dc78993a52c453f8feee75a4e8a34b2 (DEFAULT_TABLE table=test-table [id=ac4607d08ed14d259db6da39c24ddee1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 06:59:12.533821 8100 tablet_service.cc:1467] Processing CreateTablet for tablet 8dc78993a52c453f8feee75a4e8a34b2 (DEFAULT_TABLE table=test-table [id=ac4607d08ed14d259db6da39c24ddee1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 06:59:12.533828 8251 tablet_service.cc:1467] Processing CreateTablet for tablet 8dc78993a52c453f8feee75a4e8a34b2 (DEFAULT_TABLE table=test-table [id=ac4607d08ed14d259db6da39c24ddee1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 06:59:12.534921 8402 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8dc78993a52c453f8feee75a4e8a34b2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 06:59:12.534933 8100 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8dc78993a52c453f8feee75a4e8a34b2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 06:59:12.534932 8251 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8dc78993a52c453f8feee75a4e8a34b2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 06:59:12.543016 8497 tablet_bootstrap.cc:492] T 8dc78993a52c453f8feee75a4e8a34b2 P 9b9a1d0e5ee647aa95c0c14941ce99ff: Bootstrap starting.
I20260812 06:59:12.543223 8496 tablet_bootstrap.cc:492] T 8dc78993a52c453f8feee75a4e8a34b2 P 72bd459b398e4bdb9538c7ac0d7dccb2: Bootstrap starting.
I20260812 06:59:12.543524 8498 tablet_bootstrap.cc:492] T 8dc78993a52c453f8feee75a4e8a34b2 P 1ccf14d1ef9c4e4eb9a6225273c4f345: Bootstrap starting.
I20260812 06:59:12.545241 8497 tablet_bootstrap.cc:654] T 8dc78993a52c453f8feee75a4e8a34b2 P 9b9a1d0e5ee647aa95c0c14941ce99ff: Neither blocks nor log segments found. Creating new log.
I20260812 06:59:12.545498 8496 tablet_bootstrap.cc:654] T 8dc78993a52c453f8feee75a4e8a34b2 P 72bd459b398e4bdb9538c7ac0d7dccb2: Neither blocks nor log segments found. Creating new log.
I20260812 06:59:12.546065 8498 tablet_bootstrap.cc:654] T 8dc78993a52c453f8feee75a4e8a34b2 P 1ccf14d1ef9c4e4eb9a6225273c4f345: Neither blocks nor log segments found. Creating new log.
I20260812 06:59:12.546094 8497 log.cc:824] T 8dc78993a52c453f8feee75a4e8a34b2 P 9b9a1d0e5ee647aa95c0c14941ce99ff: Log is configured to *not* fsync() on all Append() calls
I20260812 06:59:12.546888 8496 log.cc:824] T 8dc78993a52c453f8feee75a4e8a34b2 P 72bd459b398e4bdb9538c7ac0d7dccb2: Log is configured to *not* fsync() on all Append() calls
I20260812 06:59:12.546887 8498 log.cc:824] T 8dc78993a52c453f8feee75a4e8a34b2 P 1ccf14d1ef9c4e4eb9a6225273c4f345: Log is configured to *not* fsync() on all Append() calls
I20260812 06:59:12.548388 8497 tablet_bootstrap.cc:492] T 8dc78993a52c453f8feee75a4e8a34b2 P 9b9a1d0e5ee647aa95c0c14941ce99ff: No bootstrap required, opened a new log
I20260812 06:59:12.548631 8497 ts_tablet_manager.cc:1397] T 8dc78993a52c453f8feee75a4e8a34b2 P 9b9a1d0e5ee647aa95c0c14941ce99ff: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260812 06:59:12.549041 8496 tablet_bootstrap.cc:492] T 8dc78993a52c453f8feee75a4e8a34b2 P 72bd459b398e4bdb9538c7ac0d7dccb2: No bootstrap required, opened a new log
I20260812 06:59:12.549224 8496 ts_tablet_manager.cc:1397] T 8dc78993a52c453f8feee75a4e8a34b2 P 72bd459b398e4bdb9538c7ac0d7dccb2: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260812 06:59:12.549324 8498 tablet_bootstrap.cc:492] T 8dc78993a52c453f8feee75a4e8a34b2 P 1ccf14d1ef9c4e4eb9a6225273c4f345: No bootstrap required, opened a new log
I20260812 06:59:12.549517 8498 ts_tablet_manager.cc:1397] T 8dc78993a52c453f8feee75a4e8a34b2 P 1ccf14d1ef9c4e4eb9a6225273c4f345: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260812 06:59:12.552372 8498 raft_consensus.cc:348] T 8dc78993a52c453f8feee75a4e8a34b2 P 1ccf14d1ef9c4e4eb9a6225273c4f345 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b9a1d0e5ee647aa95c0c14941ce99ff" member_type: VOTER last_known_addr { host: "127.19.52.65" port: 38133 } } peers { permanent_uuid: "1ccf14d1ef9c4e4eb9a6225273c4f345" member_type: VOTER last_known_addr { host: "127.19.52.66" port: 42999 } } peers { permanent_uuid: "72bd459b398e4bdb9538c7ac0d7dccb2" member_type: VOTER last_known_addr { host: "127.19.52.67" port: 33421 } }
I20260812 06:59:12.552726 8498 raft_consensus.cc:374] T 8dc78993a52c453f8feee75a4e8a34b2 P 1ccf14d1ef9c4e4eb9a6225273c4f345 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 06:59:12.552786 8498 raft_consensus.cc:729] T 8dc78993a52c453f8feee75a4e8a34b2 P 1ccf14d1ef9c4e4eb9a6225273c4f345 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1ccf14d1ef9c4e4eb9a6225273c4f345, State: Initialized, Role: FOLLOWER
I20260812 06:59:12.553272 8498 consensus_queue.cc:260] T 8dc78993a52c453f8feee75a4e8a34b2 P 1ccf14d1ef9c4e4eb9a6225273c4f345 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b9a1d0e5ee647aa95c0c14941ce99ff" member_type: VOTER last_known_addr { host: "127.19.52.65" port: 38133 } } peers { permanent_uuid: "1ccf14d1ef9c4e4eb9a6225273c4f345" member_type: VOTER last_known_addr { host: "127.19.52.66" port: 42999 } } peers { permanent_uuid: "72bd459b398e4bdb9538c7ac0d7dccb2" member_type: VOTER last_known_addr { host: "127.19.52.67" port: 33421 } }
I20260812 06:59:12.553261 8497 raft_consensus.cc:348] T 8dc78993a52c453f8feee75a4e8a34b2 P 9b9a1d0e5ee647aa95c0c14941ce99ff [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b9a1d0e5ee647aa95c0c14941ce99ff" member_type: VOTER last_known_addr { host: "127.19.52.65" port: 38133 } } peers { permanent_uuid: "1ccf14d1ef9c4e4eb9a6225273c4f345" member_type: VOTER last_known_addr { host: "127.19.52.66" port: 42999 } } peers { permanent_uuid: "72bd459b398e4bdb9538c7ac0d7dccb2" member_type: VOTER last_known_addr { host: "127.19.52.67" port: 33421 } }
I20260812 06:59:12.553261 8496 raft_consensus.cc:348] T 8dc78993a52c453f8feee75a4e8a34b2 P 72bd459b398e4bdb9538c7ac0d7dccb2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b9a1d0e5ee647aa95c0c14941ce99ff" member_type: VOTER last_known_addr { host: "127.19.52.65" port: 38133 } } peers { permanent_uuid: "1ccf14d1ef9c4e4eb9a6225273c4f345" member_type: VOTER last_known_addr { host: "127.19.52.66" port: 42999 } } peers { permanent_uuid: "72bd459b398e4bdb9538c7ac0d7dccb2" member_type: VOTER last_known_addr { host: "127.19.52.67" port: 33421 } }
I20260812 06:59:12.553521 8497 raft_consensus.cc:374] T 8dc78993a52c453f8feee75a4e8a34b2 P 9b9a1d0e5ee647aa95c0c14941ce99ff [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 06:59:12.553526 8496 raft_consensus.cc:374] T 8dc78993a52c453f8feee75a4e8a34b2 P 72bd459b398e4bdb9538c7ac0d7dccb2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 06:59:12.553583 8497 raft_consensus.cc:729] T 8dc78993a52c453f8feee75a4e8a34b2 P 9b9a1d0e5ee647aa95c0c14941ce99ff [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9b9a1d0e5ee647aa95c0c14941ce99ff, State: Initialized, Role: FOLLOWER
I20260812 06:59:12.553583 8496 raft_consensus.cc:729] T 8dc78993a52c453f8feee75a4e8a34b2 P 72bd459b398e4bdb9538c7ac0d7dccb2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 72bd459b398e4bdb9538c7ac0d7dccb2, State: Initialized, Role: FOLLOWER
I20260812 06:59:12.554015 8496 consensus_queue.cc:260] T 8dc78993a52c453f8feee75a4e8a34b2 P 72bd459b398e4bdb9538c7ac0d7dccb2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b9a1d0e5ee647aa95c0c14941ce99ff" member_type: VOTER last_known_addr { host: "127.19.52.65" port: 38133 } } peers { permanent_uuid: "1ccf14d1ef9c4e4eb9a6225273c4f345" member_type: VOTER last_known_addr { host: "127.19.52.66" port: 42999 } } peers { permanent_uuid: "72bd459b398e4bdb9538c7ac0d7dccb2" member_type: VOTER last_known_addr { host: "127.19.52.67" port: 33421 } }
I20260812 06:59:12.554082 8497 consensus_queue.cc:260] T 8dc78993a52c453f8feee75a4e8a34b2 P 9b9a1d0e5ee647aa95c0c14941ce99ff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b9a1d0e5ee647aa95c0c14941ce99ff" member_type: VOTER last_known_addr { host: "127.19.52.65" port: 38133 } } peers { permanent_uuid: "1ccf14d1ef9c4e4eb9a6225273c4f345" member_type: VOTER last_known_addr { host: "127.19.52.66" port: 42999 } } peers { permanent_uuid: "72bd459b398e4bdb9538c7ac0d7dccb2" member_type: VOTER last_known_addr { host: "127.19.52.67" port: 33421 } }
I20260812 06:59:12.554520 8498 ts_tablet_manager.cc:1428] T 8dc78993a52c453f8feee75a4e8a34b2 P 1ccf14d1ef9c4e4eb9a6225273c4f345: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260812 06:59:12.554893 8329 heartbeater.cc:503] Master 127.19.52.126:34135 was elected leader, sending a full tablet report...
I20260812 06:59:12.554998 8496 ts_tablet_manager.cc:1428] T 8dc78993a52c453f8feee75a4e8a34b2 P 72bd459b398e4bdb9538c7ac0d7dccb2: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260812 06:59:12.555389 8176 heartbeater.cc:503] Master 127.19.52.126:34135 was elected leader, sending a full tablet report...
I20260812 06:59:12.556198 8497 ts_tablet_manager.cc:1428] T 8dc78993a52c453f8feee75a4e8a34b2 P 9b9a1d0e5ee647aa95c0c14941ce99ff: Time spent starting tablet: real 0.007s user 0.006s sys 0.000s
I20260812 06:59:12.556293 8476 heartbeater.cc:503] Master 127.19.52.126:34135 was elected leader, sending a full tablet report...
I20260812 06:59:12.638861 8505 raft_consensus.cc:482] T 8dc78993a52c453f8feee75a4e8a34b2 P 1ccf14d1ef9c4e4eb9a6225273c4f345 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 06:59:12.639128 8505 raft_consensus.cc:504] T 8dc78993a52c453f8feee75a4e8a34b2 P 1ccf14d1ef9c4e4eb9a6225273c4f345 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b9a1d0e5ee647aa95c0c14941ce99ff" member_type: VOTER last_known_addr { host: "127.19.52.65" port: 38133 } } peers { permanent_uuid: "1ccf14d1ef9c4e4eb9a6225273c4f345" member_type: VOTER last_known_addr { host: "127.19.52.66" port: 42999 } } peers { permanent_uuid: "72bd459b398e4bdb9538c7ac0d7dccb2" member_type: VOTER last_known_addr { host: "127.19.52.67" port: 33421 } }
I20260812 06:59:12.640381 8505 leader_election.cc:288] T 8dc78993a52c453f8feee75a4e8a34b2 P 1ccf14d1ef9c4e4eb9a6225273c4f345 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9b9a1d0e5ee647aa95c0c14941ce99ff (127.19.52.65:38133), 72bd459b398e4bdb9538c7ac0d7dccb2 (127.19.52.67:33421)
I20260812 06:59:12.644426 8488 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:59:12.640786 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.52.66:42953 (local address 127.19.52.65:38133)
0812 06:59:12.640910 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0812 06:59:12.640914 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 06:59:12.640936 (+ 22us) server_negotiation.cc:407] Connection header received
0812 06:59:12.641099 (+ 163us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:59:12.641103 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:59:12.641179 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:59:12.641259 (+ 80us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 06:59:12.641634 (+ 375us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:59:12.642221 (+ 587us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:59:12.643358 (+ 1137us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:59:12.643806 (+ 448us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:59:12.643903 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:59:12.643989 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0812 06:59:12.644209 (+ 220us) server_negotiation.cc:299] Negotiation successful
0812 06:59:12.644289 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260812 06:59:12.644441 8508 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:59:12.640754 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.67:33421 (local address 127.19.52.66:54319)
0812 06:59:12.641104 (+ 350us) negotiation.cc:107] Waiting for socket to connect
0812 06:59:12.641116 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 06:59:12.641213 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:59:12.641504 (+ 291us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:59:12.641507 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:59:12.641521 (+ 14us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 06:59:12.641674 (+ 153us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:59:12.641681 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:59:12.642468 (+ 787us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:59:12.642470 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:59:12.643272 (+ 802us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:59:12.643306 (+ 34us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:59:12.643998 (+ 692us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:59:12.644004 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:59:12.644099 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:59:12.644116 (+ 17us) client_negotiation.cc:769] Sending connection context
0812 06:59:12.644216 (+ 100us) client_negotiation.cc:240] Negotiation successful
0812 06:59:12.644288 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":292,"thread_start_us":156,"threads_started":1}
I20260812 06:59:12.644443 8495 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:59:12.640829 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.52.66:54319 (local address 127.19.52.67:33421)
0812 06:59:12.640994 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0812 06:59:12.641036 (+ 42us) server_negotiation.cc:399] Waiting for connection header
0812 06:59:12.641255 (+ 219us) server_negotiation.cc:407] Connection header received
0812 06:59:12.641330 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:59:12.641334 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:59:12.641380 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:59:12.641487 (+ 107us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 06:59:12.641795 (+ 308us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:59:12.642350 (+ 555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:59:12.643432 (+ 1082us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:59:12.643834 (+ 402us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:59:12.644001 (+ 167us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:59:12.644053 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0812 06:59:12.644251 (+ 198us) server_negotiation.cc:299] Negotiation successful
0812 06:59:12.644329 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260812 06:59:12.644445 8331 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:59:12.640632 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.65:38133 (local address 127.19.52.66:42953)
0812 06:59:12.640824 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0812 06:59:12.640841 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0812 06:59:12.640953 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:59:12.641318 (+ 365us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:59:12.641322 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:59:12.641348 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 06:59:12.641502 (+ 154us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:59:12.641510 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:59:12.642347 (+ 837us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:59:12.642350 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:59:12.643215 (+ 865us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:59:12.643225 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:59:12.643941 (+ 716us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:59:12.643946 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:59:12.644054 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:59:12.644085 (+ 31us) client_negotiation.cc:769] Sending connection context
0812 06:59:12.644162 (+ 77us) client_negotiation.cc:240] Negotiation successful
0812 06:59:12.644261 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":111}
I20260812 06:59:12.645159 8426 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8dc78993a52c453f8feee75a4e8a34b2" candidate_uuid: "1ccf14d1ef9c4e4eb9a6225273c4f345" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "72bd459b398e4bdb9538c7ac0d7dccb2" is_pre_election: true
I20260812 06:59:12.645153 8124 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8dc78993a52c453f8feee75a4e8a34b2" candidate_uuid: "1ccf14d1ef9c4e4eb9a6225273c4f345" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9b9a1d0e5ee647aa95c0c14941ce99ff" is_pre_election: true
I20260812 06:59:12.645499 8124 raft_consensus.cc:2454] T 8dc78993a52c453f8feee75a4e8a34b2 P 9b9a1d0e5ee647aa95c0c14941ce99ff [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ccf14d1ef9c4e4eb9a6225273c4f345 in term 0.
I20260812 06:59:12.645499 8426 raft_consensus.cc:2454] T 8dc78993a52c453f8feee75a4e8a34b2 P 72bd459b398e4bdb9538c7ac0d7dccb2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ccf14d1ef9c4e4eb9a6225273c4f345 in term 0.
I20260812 06:59:12.646031 8200 leader_election.cc:302] T 8dc78993a52c453f8feee75a4e8a34b2 P 1ccf14d1ef9c4e4eb9a6225273c4f345 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ccf14d1ef9c4e4eb9a6225273c4f345, 72bd459b398e4bdb9538c7ac0d7dccb2; no voters:
I20260812 06:59:12.646327 8505 raft_consensus.cc:2781] T 8dc78993a52c453f8feee75a4e8a34b2 P 1ccf14d1ef9c4e4eb9a6225273c4f345 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 06:59:12.646431 8505 raft_consensus.cc:482] T 8dc78993a52c453f8feee75a4e8a34b2 P 1ccf14d1ef9c4e4eb9a6225273c4f345 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 06:59:12.646495 8505 raft_consensus.cc:3037] T 8dc78993a52c453f8feee75a4e8a34b2 P 1ccf14d1ef9c4e4eb9a6225273c4f345 [term 0 FOLLOWER]: Advancing to term 1
I20260812 06:59:12.647585 8505 raft_consensus.cc:504] T 8dc78993a52c453f8feee75a4e8a34b2 P 1ccf14d1ef9c4e4eb9a6225273c4f345 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b9a1d0e5ee647aa95c0c14941ce99ff" member_type: VOTER last_known_addr { host: "127.19.52.65" port: 38133 } } peers { permanent_uuid: "1ccf14d1ef9c4e4eb9a6225273c4f345" member_type: VOTER last_known_addr { host: "127.19.52.66" port: 42999 } } peers { permanent_uuid: "72bd459b398e4bdb9538c7ac0d7dccb2" member_type: VOTER last_known_addr { host: "127.19.52.67" port: 33421 } }
I20260812 06:59:12.648000 8505 leader_election.cc:288] T 8dc78993a52c453f8feee75a4e8a34b2 P 1ccf14d1ef9c4e4eb9a6225273c4f345 [CANDIDATE]: Term 1 election: Requested vote from peers 9b9a1d0e5ee647aa95c0c14941ce99ff (127.19.52.65:38133), 72bd459b398e4bdb9538c7ac0d7dccb2 (127.19.52.67:33421)
I20260812 06:59:12.648463 8426 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8dc78993a52c453f8feee75a4e8a34b2" candidate_uuid: "1ccf14d1ef9c4e4eb9a6225273c4f345" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "72bd459b398e4bdb9538c7ac0d7dccb2"
I20260812 06:59:12.648494 8124 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8dc78993a52c453f8feee75a4e8a34b2" candidate_uuid: "1ccf14d1ef9c4e4eb9a6225273c4f345" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9b9a1d0e5ee647aa95c0c14941ce99ff"
I20260812 06:59:12.648624 8426 raft_consensus.cc:3037] T 8dc78993a52c453f8feee75a4e8a34b2 P 72bd459b398e4bdb9538c7ac0d7dccb2 [term 0 FOLLOWER]: Advancing to term 1
I20260812 06:59:12.648699 8124 raft_consensus.cc:3037] T 8dc78993a52c453f8feee75a4e8a34b2 P 9b9a1d0e5ee647aa95c0c14941ce99ff [term 0 FOLLOWER]: Advancing to term 1
I20260812 06:59:12.649925 8426 raft_consensus.cc:2454] T 8dc78993a52c453f8feee75a4e8a34b2 P 72bd459b398e4bdb9538c7ac0d7dccb2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1ccf14d1ef9c4e4eb9a6225273c4f345 in term 1.
I20260812 06:59:12.650027 8124 raft_consensus.cc:2454] T 8dc78993a52c453f8feee75a4e8a34b2 P 9b9a1d0e5ee647aa95c0c14941ce99ff [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1ccf14d1ef9c4e4eb9a6225273c4f345 in term 1.
I20260812 06:59:12.650291 8200 leader_election.cc:302] T 8dc78993a52c453f8feee75a4e8a34b2 P 1ccf14d1ef9c4e4eb9a6225273c4f345 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ccf14d1ef9c4e4eb9a6225273c4f345, 72bd459b398e4bdb9538c7ac0d7dccb2; no voters:
I20260812 06:59:12.650472 8505 raft_consensus.cc:2781] T 8dc78993a52c453f8feee75a4e8a34b2 P 1ccf14d1ef9c4e4eb9a6225273c4f345 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 06:59:12.650729 8505 raft_consensus.cc:686] T 8dc78993a52c453f8feee75a4e8a34b2 P 1ccf14d1ef9c4e4eb9a6225273c4f345 [term 1 LEADER]: Becoming Leader. State: Replica: 1ccf14d1ef9c4e4eb9a6225273c4f345, State: Running, Role: LEADER
I20260812 06:59:12.651091 8505 consensus_queue.cc:237] T 8dc78993a52c453f8feee75a4e8a34b2 P 1ccf14d1ef9c4e4eb9a6225273c4f345 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b9a1d0e5ee647aa95c0c14941ce99ff" member_type: VOTER last_known_addr { host: "127.19.52.65" port: 38133 } } peers { permanent_uuid: "1ccf14d1ef9c4e4eb9a6225273c4f345" member_type: VOTER last_known_addr { host: "127.19.52.66" port: 42999 } } peers { permanent_uuid: "72bd459b398e4bdb9538c7ac0d7dccb2" member_type: VOTER last_known_addr { host: "127.19.52.67" port: 33421 } }
I20260812 06:59:12.654534 7945 catalog_manager.cc:5495] T 8dc78993a52c453f8feee75a4e8a34b2 P 1ccf14d1ef9c4e4eb9a6225273c4f345 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1ccf14d1ef9c4e4eb9a6225273c4f345 (127.19.52.66). New cstate: current_term: 1 leader_uuid: "1ccf14d1ef9c4e4eb9a6225273c4f345" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b9a1d0e5ee647aa95c0c14941ce99ff" member_type: VOTER last_known_addr { host: "127.19.52.65" port: 38133 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1ccf14d1ef9c4e4eb9a6225273c4f345" member_type: VOTER last_known_addr { host: "127.19.52.66" port: 42999 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "72bd459b398e4bdb9538c7ac0d7dccb2" member_type: VOTER last_known_addr { host: "127.19.52.67" port: 33421 } health_report { overall_health: UNKNOWN } } }
I20260812 06:59:12.688422 8492 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:59:12.684487 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59698 (local address 127.19.52.66:42999)
0812 06:59:12.684661 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0812 06:59:12.684665 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 06:59:12.684680 (+ 15us) server_negotiation.cc:407] Connection header received
0812 06:59:12.684804 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:59:12.684806 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:59:12.684865 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:59:12.684958 (+ 93us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 06:59:12.685238 (+ 280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:59:12.685771 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:59:12.686677 (+ 906us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:59:12.686925 (+ 248us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:59:12.687036 (+ 111us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:59:12.687352 (+ 316us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 06:59:12.687842 (+ 490us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 06:59:12.687973 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0812 06:59:12.688150 (+ 177us) server_negotiation.cc:299] Negotiation successful
0812 06:59:12.688200 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
I20260812 06:59:12.694573 8124 raft_consensus.cc:1261] T 8dc78993a52c453f8feee75a4e8a34b2 P 9b9a1d0e5ee647aa95c0c14941ce99ff [term 1 FOLLOWER]: Refusing update from remote peer 1ccf14d1ef9c4e4eb9a6225273c4f345: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 06:59:12.694573 8426 raft_consensus.cc:1261] T 8dc78993a52c453f8feee75a4e8a34b2 P 72bd459b398e4bdb9538c7ac0d7dccb2 [term 1 FOLLOWER]: Refusing update from remote peer 1ccf14d1ef9c4e4eb9a6225273c4f345: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 06:59:12.695370 8505 consensus_queue.cc:1035] T 8dc78993a52c453f8feee75a4e8a34b2 P 1ccf14d1ef9c4e4eb9a6225273c4f345 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9b9a1d0e5ee647aa95c0c14941ce99ff" member_type: VOTER last_known_addr { host: "127.19.52.65" port: 38133 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 06:59:12.695585 8509 consensus_queue.cc:1035] T 8dc78993a52c453f8feee75a4e8a34b2 P 1ccf14d1ef9c4e4eb9a6225273c4f345 [LEADER]: Connected to new peer: Peer: permanent_uuid: "72bd459b398e4bdb9538c7ac0d7dccb2" member_type: VOTER last_known_addr { host: "127.19.52.67" port: 33421 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 06:59:12.702692 8512 mvcc.cc:205] Tried to move back new op lower bound from 7317577534229241856 to 7317577534061932544. Current Snapshot: MvccSnapshot[applied={T|T < 7317577534229241856}]
I20260812 06:59:12.705183 8513 mvcc.cc:205] Tried to move back new op lower bound from 7317577534229241856 to 7317577534061932544. Current Snapshot: MvccSnapshot[applied={T|T < 7317577534229241856}]
I20260812 06:59:12.712797 7946 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57588:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 06:59:12.713057 7946 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57588:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 06:59:12.715794 7946 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 47b0874b048b4c79975ebd25f1fab9d7: Sending DeleteTablet for 3 replicas of tablet 8dc78993a52c453f8feee75a4e8a34b2
I20260812 06:59:12.716753 8251 tablet_service.cc:1514] Processing DeleteTablet for tablet 8dc78993a52c453f8feee75a4e8a34b2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 06:59:12 UTC) from {username='kudu', principal='kudu/127.19.52.126@KRBTEST.COM'} at 127.0.0.1:59692
I20260812 06:59:12.716750 8100 tablet_service.cc:1514] Processing DeleteTablet for tablet 8dc78993a52c453f8feee75a4e8a34b2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 06:59:12 UTC) from {username='kudu', principal='kudu/127.19.52.126@KRBTEST.COM'} at 127.0.0.1:44362
I20260812 06:59:12.717233 8402 tablet_service.cc:1514] Processing DeleteTablet for tablet 8dc78993a52c453f8feee75a4e8a34b2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 06:59:12 UTC) from {username='kudu', principal='kudu/127.19.52.126@KRBTEST.COM'} at 127.0.0.1:53298
I20260812 06:59:12.717355 8523 tablet_replica.cc:330] T 8dc78993a52c453f8feee75a4e8a34b2 P 9b9a1d0e5ee647aa95c0c14941ce99ff: stopping tablet replica
I20260812 06:59:12.717720 8523 raft_consensus.cc:2229] T 8dc78993a52c453f8feee75a4e8a34b2 P 9b9a1d0e5ee647aa95c0c14941ce99ff [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 06:59:12.717756 8525 tablet_replica.cc:330] T 8dc78993a52c453f8feee75a4e8a34b2 P 72bd459b398e4bdb9538c7ac0d7dccb2: stopping tablet replica
I20260812 06:59:12.718086 8523 raft_consensus.cc:2258] T 8dc78993a52c453f8feee75a4e8a34b2 P 9b9a1d0e5ee647aa95c0c14941ce99ff [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 06:59:12.718225 8524 tablet_replica.cc:330] T 8dc78993a52c453f8feee75a4e8a34b2 P 1ccf14d1ef9c4e4eb9a6225273c4f345: stopping tablet replica
I20260812 06:59:12.718518 8524 raft_consensus.cc:2229] T 8dc78993a52c453f8feee75a4e8a34b2 P 1ccf14d1ef9c4e4eb9a6225273c4f345 [term 1 LEADER]: Raft consensus shutting down.
I20260812 06:59:12.718977 8524 raft_consensus.cc:2258] T 8dc78993a52c453f8feee75a4e8a34b2 P 1ccf14d1ef9c4e4eb9a6225273c4f345 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 06:59:12.719264 19665 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu with pid 8020
I20260812 06:59:12.725389 8525 raft_consensus.cc:2229] T 8dc78993a52c453f8feee75a4e8a34b2 P 72bd459b398e4bdb9538c7ac0d7dccb2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 06:59:12.725795 8525 raft_consensus.cc:2258] T 8dc78993a52c453f8feee75a4e8a34b2 P 72bd459b398e4bdb9538c7ac0d7dccb2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 06:59:12.726475 8524 ts_tablet_manager.cc:1905] T 8dc78993a52c453f8feee75a4e8a34b2 P 1ccf14d1ef9c4e4eb9a6225273c4f345: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 06:59:12.729449 8524 ts_tablet_manager.cc:1918] T 8dc78993a52c453f8feee75a4e8a34b2 P 1ccf14d1ef9c4e4eb9a6225273c4f345: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 06:59:12.729566 8524 log.cc:1196] T 8dc78993a52c453f8feee75a4e8a34b2 P 1ccf14d1ef9c4e4eb9a6225273c4f345: Deleting WAL directory at /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786517687384214-19665-0/minicluster-data/ts-1/wal/wals/8dc78993a52c453f8feee75a4e8a34b2
I20260812 06:59:12.729903 8524 ts_tablet_manager.cc:1939] T 8dc78993a52c453f8feee75a4e8a34b2 P 1ccf14d1ef9c4e4eb9a6225273c4f345: Deleting consensus metadata
W20260812 06:59:12.730690 7932 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.19.52.65:38133 (error 108)
I20260812 06:59:12.731359 7930 catalog_manager.cc:4841] TS 1ccf14d1ef9c4e4eb9a6225273c4f345 (127.19.52.66:42999): tablet 8dc78993a52c453f8feee75a4e8a34b2 (table test-table [id=ac4607d08ed14d259db6da39c24ddee1]) successfully deleted
I20260812 06:59:12.731696 19665 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu with pid 8181
I20260812 06:59:12.732436 8489 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:59:12.732022 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.65:38133 (local address 127.0.0.1:44370)
0812 06:59:12.732236 (+ 214us) negotiation.cc:107] Waiting for socket to connect
0812 06:59:12.732319 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.52.65:38133: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":110}
W20260812 06:59:12.732731 7932 catalog_manager.cc:4584] TS 9b9a1d0e5ee647aa95c0c14941ce99ff (127.19.52.65:38133): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 8dc78993a52c453f8feee75a4e8a34b2: Network error: Client connection negotiation failed: client connection to 127.19.52.65:38133: connect: Connection refused (error 111)
I20260812 06:59:12.735483 8525 ts_tablet_manager.cc:1905] T 8dc78993a52c453f8feee75a4e8a34b2 P 72bd459b398e4bdb9538c7ac0d7dccb2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 06:59:12.739490 8525 ts_tablet_manager.cc:1918] T 8dc78993a52c453f8feee75a4e8a34b2 P 72bd459b398e4bdb9538c7ac0d7dccb2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 06:59:12.739653 8525 log.cc:1196] T 8dc78993a52c453f8feee75a4e8a34b2 P 72bd459b398e4bdb9538c7ac0d7dccb2: Deleting WAL directory at /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786517687384214-19665-0/minicluster-data/ts-2/wal/wals/8dc78993a52c453f8feee75a4e8a34b2
I20260812 06:59:12.740022 8525 ts_tablet_manager.cc:1939] T 8dc78993a52c453f8feee75a4e8a34b2 P 72bd459b398e4bdb9538c7ac0d7dccb2: Deleting consensus metadata
I20260812 06:59:12.741683 7929 catalog_manager.cc:4841] TS 72bd459b398e4bdb9538c7ac0d7dccb2 (127.19.52.67:33421): tablet 8dc78993a52c453f8feee75a4e8a34b2 (table test-table [id=ac4607d08ed14d259db6da39c24ddee1]) successfully deleted
I20260812 06:59:12.742955 19665 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu with pid 8335
I20260812 06:59:12.751004 19665 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu with pid 7910
2026-08-12T06:59:12Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/2 (2580 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/3
2026-08-12T06:59:12Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T06:59:12Z Disabled control of system clock
I20260812 06:59:12.788764 19665 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu
/tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786517687384214-19665-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786517687384214-19665-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786517687384214-19665-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786517687384214-19665-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.52.126:45605
--webserver_interface=127.19.52.126
--webserver_port=0
--builtin_ntp_servers=127.19.52.84:34387
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.19.52.126:45605
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260812 06:59:12.904798 8530 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:59:12.905095 8530 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:59:12.905135 8530 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 06:59:12.909273 8530 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 06:59:12.909343 8530 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 06:59:12.909358 8530 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 06:59:12.909375 8530 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 06:59:12.909390 8530 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 06:59:12.914265 8530 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.52.84:34387
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786517687384214-19665-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786517687384214-19665-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.19.52.126:45605
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.52.126:45605
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786517687384214-19665-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.19.52.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8530
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786517687384214-19665-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 06:15:54 UTC on ccycloud.quasar-rnfzce.root.comops.site
build id 4581989
I20260812 06:59:12.915438 8530 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:59:12.916805 8530 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 06:59:12.925722 8538 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:59:12.925714 8537 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:59:12.926024 8530 server_base.cc:1034] running on GCE node
W20260812 06:59:12.925714 8540 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:59:12.926684 8530 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:59:12.927837 8530 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:59:12.929040 8530 hybrid_clock.cc:648] HybridClock initialized: now 1786517952929016 us; error 36 us; skew 500 ppm
I20260812 06:59:12.935904 8530 webserver.cc:460] Webserver started at http://127.19.52.126:40077/ using document root <none> and password file <none>
I20260812 06:59:12.936632 8530 fs_manager.cc:359] Metadata directory not provided
I20260812 06:59:12.936736 8530 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:59:12.937005 8530 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 06:59:12.939218 8530 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786517687384214-19665-0/minicluster-data/master-0/data/instance:
uuid: "d91f95e589d044688c362db5a0edeb96"
format_stamp: "Formatted at 2026-08-12 06:59:12 on dist-test-slave-65mx"
server_key: "7b944b3f221cba94527154e2348e5b7e"
server_key_iv: "29d1bdc23e744cd558c58ca56e03331a"
server_key_version: "encryptionkey@0"
I20260812 06:59:12.939831 8530 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786517687384214-19665-0/minicluster-data/master-0/wal/instance:
uuid: "d91f95e589d044688c362db5a0edeb96"
format_stamp: "Formatted at 2026-08-12 06:59:12 on dist-test-slave-65mx"
server_key: "7b944b3f221cba94527154e2348e5b7e"
server_key_iv: "29d1bdc23e744cd558c58ca56e03331a"
server_key_version: "encryptionkey@0"
I20260812 06:59:12.943825 8530 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260812 06:59:12.946641 8547 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 06:59:12.948011 8530 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260812 06:59:12.948191 8530 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786517687384214-19665-0/minicluster-data/master-0/data,/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786517687384214-19665-0/minicluster-data/master-0/wal
uuid: "d91f95e589d044688c362db5a0edeb96"
format_stamp: "Formatted at 2026-08-12 06:59:12 on dist-test-slave-65mx"
server_key: "7b944b3f221cba94527154e2348e5b7e"
server_key_iv: "29d1bdc23e744cd558c58ca56e03331a"
server_key_version: "encryptionkey@0"
I20260812 06:59:12.948326 8530 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786517687384214-19665-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786517687384214-19665-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786517687384214-19665-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:59:12.971761 8530 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 06:59:12.972626 8530 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 06:59:12.972843 8530 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:59:12.980996 8530 rpc_server.cc:307] RPC server started. Bound to: 127.19.52.126:45605
I20260812 06:59:12.981012 8610 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.52.126:45605 every 8 connection(s)
I20260812 06:59:12.982050 8530 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786517687384214-19665-0/minicluster-data/master-0/data/info.pb
I20260812 06:59:12.985383 8611 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 06:59:12.985985 19665 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu as pid 8530
I20260812 06:59:12.986179 19665 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786517687384214-19665-0/minicluster-data/master-0/wal/instance
I20260812 06:59:12.986608 19665 external_mini_cluster.cc:1428] Setting key 51be6115083690be785b7ec81ea47154
I20260812 06:59:12.994069 8611 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d91f95e589d044688c362db5a0edeb96: Bootstrap starting.
I20260812 06:59:12.996910 8611 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d91f95e589d044688c362db5a0edeb96: Neither blocks nor log segments found. Creating new log.
I20260812 06:59:12.997249 8614 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:59:12.988245 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49446 (local address 127.19.52.126:45605)
0812 06:59:12.988942 (+ 697us) server_negotiation.cc:206] Beginning negotiation
0812 06:59:12.988958 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0812 06:59:12.988996 (+ 38us) server_negotiation.cc:407] Connection header received
0812 06:59:12.989886 (+ 890us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:59:12.989906 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:59:12.990303 (+ 397us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:59:12.990615 (+ 312us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:59:12.990986 (+ 371us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:59:12.991953 (+ 967us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:59:12.993347 (+ 1394us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:59:12.993865 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:59:12.993993 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:59:12.995091 (+ 1098us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:59:12.995097 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:59:12.995110 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 06:59:12.995574 (+ 464us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:59:12.995639 (+ 65us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 06:59:12.995662 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:59:12.995757 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0812 06:59:12.995983 (+ 226us) server_negotiation.cc:299] Negotiation successful
0812 06:59:12.996345 (+ 362us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":344,"thread_start_us":145,"threads_started":1}
I20260812 06:59:12.997864 8611 log.cc:824] T 00000000000000000000000000000000 P d91f95e589d044688c362db5a0edeb96: Log is configured to *not* fsync() on all Append() calls
I20260812 06:59:13.000021 8611 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d91f95e589d044688c362db5a0edeb96: No bootstrap required, opened a new log
I20260812 06:59:13.003988 8611 raft_consensus.cc:348] T 00000000000000000000000000000000 P d91f95e589d044688c362db5a0edeb96 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d91f95e589d044688c362db5a0edeb96" member_type: VOTER last_known_addr { host: "127.19.52.126" port: 45605 } }
I20260812 06:59:13.004253 8611 raft_consensus.cc:374] T 00000000000000000000000000000000 P d91f95e589d044688c362db5a0edeb96 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 06:59:13.004324 8611 raft_consensus.cc:729] T 00000000000000000000000000000000 P d91f95e589d044688c362db5a0edeb96 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d91f95e589d044688c362db5a0edeb96, State: Initialized, Role: FOLLOWER
I20260812 06:59:13.004853 8611 consensus_queue.cc:260] T 00000000000000000000000000000000 P d91f95e589d044688c362db5a0edeb96 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d91f95e589d044688c362db5a0edeb96" member_type: VOTER last_known_addr { host: "127.19.52.126" port: 45605 } }
I20260812 06:59:13.005005 8611 raft_consensus.cc:388] T 00000000000000000000000000000000 P d91f95e589d044688c362db5a0edeb96 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 06:59:13.005113 8611 raft_consensus.cc:482] T 00000000000000000000000000000000 P d91f95e589d044688c362db5a0edeb96 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 06:59:13.005262 8611 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d91f95e589d044688c362db5a0edeb96 [term 0 FOLLOWER]: Advancing to term 1
I20260812 06:59:13.006350 8611 raft_consensus.cc:504] T 00000000000000000000000000000000 P d91f95e589d044688c362db5a0edeb96 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d91f95e589d044688c362db5a0edeb96" member_type: VOTER last_known_addr { host: "127.19.52.126" port: 45605 } }
I20260812 06:59:13.006726 8611 leader_election.cc:302] T 00000000000000000000000000000000 P d91f95e589d044688c362db5a0edeb96 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d91f95e589d044688c362db5a0edeb96; no voters:
I20260812 06:59:13.007094 8611 leader_election.cc:288] T 00000000000000000000000000000000 P d91f95e589d044688c362db5a0edeb96 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 06:59:13.007241 8616 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d91f95e589d044688c362db5a0edeb96 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 06:59:13.007504 8616 raft_consensus.cc:686] T 00000000000000000000000000000000 P d91f95e589d044688c362db5a0edeb96 [term 1 LEADER]: Becoming Leader. State: Replica: d91f95e589d044688c362db5a0edeb96, State: Running, Role: LEADER
I20260812 06:59:13.007833 8616 consensus_queue.cc:237] T 00000000000000000000000000000000 P d91f95e589d044688c362db5a0edeb96 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d91f95e589d044688c362db5a0edeb96" member_type: VOTER last_known_addr { host: "127.19.52.126" port: 45605 } }
I20260812 06:59:13.008646 8611 sys_catalog.cc:564] T 00000000000000000000000000000000 P d91f95e589d044688c362db5a0edeb96 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 06:59:13.009765 8618 sys_catalog.cc:455] T 00000000000000000000000000000000 P d91f95e589d044688c362db5a0edeb96 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d91f95e589d044688c362db5a0edeb96" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d91f95e589d044688c362db5a0edeb96" member_type: VOTER last_known_addr { host: "127.19.52.126" port: 45605 } } }
I20260812 06:59:13.009792 8619 sys_catalog.cc:455] T 00000000000000000000000000000000 P d91f95e589d044688c362db5a0edeb96 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d91f95e589d044688c362db5a0edeb96. Latest consensus state: current_term: 1 leader_uuid: "d91f95e589d044688c362db5a0edeb96" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d91f95e589d044688c362db5a0edeb96" member_type: VOTER last_known_addr { host: "127.19.52.126" port: 45605 } } }
I20260812 06:59:13.009969 8619 sys_catalog.cc:458] T 00000000000000000000000000000000 P d91f95e589d044688c362db5a0edeb96 [sys.catalog]: This master's current role is: LEADER
I20260812 06:59:13.010211 8618 sys_catalog.cc:458] T 00000000000000000000000000000000 P d91f95e589d044688c362db5a0edeb96 [sys.catalog]: This master's current role is: LEADER
I20260812 06:59:13.010387 8626 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 06:59:13.013443 8626 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 06:59:13.019557 8626 catalog_manager.cc:1347] Generated new cluster ID: 824d1dcba54f4728805eca285932e9a5
I20260812 06:59:13.019646 8626 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 06:59:13.074206 8626 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 06:59:13.075655 8626 catalog_manager.cc:1504] Loading token signing keys...
I20260812 06:59:13.085711 8626 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d91f95e589d044688c362db5a0edeb96: Generated new TSK 0
I20260812 06:59:13.086660 8626 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 06:59:13.099632 19665 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu
/tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786517687384214-19665-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786517687384214-19665-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786517687384214-19665-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786517687384214-19665-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.52.65:0
--local_ip_for_outbound_sockets=127.19.52.65
--webserver_interface=127.19.52.65
--webserver_port=0
--tserver_master_addrs=127.19.52.126:45605
--builtin_ntp_servers=127.19.52.84:34387
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260812 06:59:13.218833 8639 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:59:13.219117 8639 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:59:13.219157 8639 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 06:59:13.223425 8639 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 06:59:13.223493 8639 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 06:59:13.223551 8639 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.52.65
I20260812 06:59:13.228835 8639 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.52.84:34387
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786517687384214-19665-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786517687384214-19665-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.52.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786517687384214-19665-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.19.52.65
--webserver_port=0
--tserver_master_addrs=127.19.52.126:45605
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8639
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.52.65
--log_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786517687384214-19665-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 06:15:54 UTC on ccycloud.quasar-rnfzce.root.comops.site
build id 4581989
I20260812 06:59:13.229959 8639 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:59:13.231315 8639 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 06:59:13.241293 8645 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:59:13.241312 8648 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:59:13.241369 8639 server_base.cc:1034] running on GCE node
W20260812 06:59:13.241312 8646 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:59:13.242012 8639 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:59:13.242722 8639 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:59:13.243916 8639 hybrid_clock.cc:648] HybridClock initialized: now 1786517953243901 us; error 34 us; skew 500 ppm
I20260812 06:59:13.245900 8639 webserver.cc:460] Webserver started at http://127.19.52.65:37957/ using document root <none> and password file <none>
I20260812 06:59:13.246541 8639 fs_manager.cc:359] Metadata directory not provided
I20260812 06:59:13.246641 8639 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:59:13.246899 8639 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 06:59:13.249054 8639 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786517687384214-19665-0/minicluster-data/ts-0/data/instance:
uuid: "7d62bb78c05d46b4a31f3925558aac7d"
format_stamp: "Formatted at 2026-08-12 06:59:13 on dist-test-slave-65mx"
server_key: "5257fe2463988f241dcf3f10aecf63dc"
server_key_iv: "bced7640c4ef3f757017fd4f7addfd80"
server_key_version: "encryptionkey@0"
I20260812 06:59:13.249640 8639 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786517687384214-19665-0/minicluster-data/ts-0/wal/instance:
uuid: "7d62bb78c05d46b4a31f3925558aac7d"
format_stamp: "Formatted at 2026-08-12 06:59:13 on dist-test-slave-65mx"
server_key: "5257fe2463988f241dcf3f10aecf63dc"
server_key_iv: "bced7640c4ef3f757017fd4f7addfd80"
server_key_version: "encryptionkey@0"
I20260812 06:59:13.253607 8639 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260812 06:59:13.256083 8654 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 06:59:13.257339 8639 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260812 06:59:13.257476 8639 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786517687384214-19665-0/minicluster-data/ts-0/data,/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786517687384214-19665-0/minicluster-data/ts-0/wal
uuid: "7d62bb78c05d46b4a31f3925558aac7d"
format_stamp: "Formatted at 2026-08-12 06:59:13 on dist-test-slave-65mx"
server_key: "5257fe2463988f241dcf3f10aecf63dc"
server_key_iv: "bced7640c4ef3f757017fd4f7addfd80"
server_key_version: "encryptionkey@0"
I20260812 06:59:13.257599 8639 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786517687384214-19665-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786517687384214-19665-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786517687384214-19665-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:59:13.270430 8639 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 06:59:13.271145 8639 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 06:59:13.271354 8639 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:59:13.271958 8639 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 06:59:13.273008 8639 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 06:59:13.273082 8639 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 06:59:13.273152 8639 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 06:59:13.273201 8639 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 06:59:13.282918 8639 rpc_server.cc:307] RPC server started. Bound to: 127.19.52.65:41247
I20260812 06:59:13.282953 8785 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.52.65:41247 every 8 connection(s)
I20260812 06:59:13.283882 8639 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786517687384214-19665-0/minicluster-data/ts-0/data/info.pb
I20260812 06:59:13.287797 19665 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu as pid 8639
I20260812 06:59:13.287945 19665 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786517687384214-19665-0/minicluster-data/ts-0/wal/instance
I20260812 06:59:13.288309 19665 external_mini_cluster.cc:1428] Setting key 787dd40e49b2a50e37e5153a84e549f6
I20260812 06:59:13.290678 19665 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu
/tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786517687384214-19665-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786517687384214-19665-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786517687384214-19665-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786517687384214-19665-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.52.66:0
--local_ip_for_outbound_sockets=127.19.52.66
--webserver_interface=127.19.52.66
--webserver_port=0
--tserver_master_addrs=127.19.52.126:45605
--builtin_ntp_servers=127.19.52.84:34387
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260812 06:59:13.294759 8614 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:59:13.285931 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.52.65:38603 (local address 127.19.52.126:45605)
0812 06:59:13.286109 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0812 06:59:13.286113 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 06:59:13.287229 (+ 1116us) server_negotiation.cc:407] Connection header received
0812 06:59:13.288944 (+ 1715us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:59:13.288948 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:59:13.289006 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:59:13.289113 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:59:13.290316 (+ 1203us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:59:13.291117 (+ 801us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:59:13.292324 (+ 1207us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:59:13.292693 (+ 369us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:59:13.292780 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:59:13.294152 (+ 1372us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:59:13.294158 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:59:13.294164 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 06:59:13.294183 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:59:13.294209 (+ 26us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 06:59:13.294222 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:59:13.294304 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0812 06:59:13.294455 (+ 151us) server_negotiation.cc:299] Negotiation successful
0812 06:59:13.294540 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260812 06:59:13.295538 8788 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:59:13.286311 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.126:45605 (local address 127.19.52.65:38603)
0812 06:59:13.287058 (+ 747us) negotiation.cc:107] Waiting for socket to connect
0812 06:59:13.287101 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0812 06:59:13.288657 (+ 1556us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:59:13.289322 (+ 665us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:59:13.289330 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:59:13.289436 (+ 106us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:59:13.290156 (+ 720us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:59:13.290174 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:59:13.291266 (+ 1092us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:59:13.291269 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:59:13.292193 (+ 924us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:59:13.292202 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:59:13.292861 (+ 659us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:59:13.292867 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:59:13.293136 (+ 269us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:59:13.293831 (+ 695us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 06:59:13.293836 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:59:13.293887 (+ 51us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 06:59:13.293978 (+ 91us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 06:59:13.294309 (+ 331us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 06:59:13.294316 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 06:59:13.294321 (+ 5us) client_negotiation.cc:769] Sending connection context
0812 06:59:13.294421 (+ 100us) client_negotiation.cc:240] Negotiation successful
0812 06:59:13.294648 (+ 227us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":303,"thread_start_us":136,"threads_started":1}
I20260812 06:59:13.296962 8786 heartbeater.cc:347] Connected to a master server at 127.19.52.126:45605
I20260812 06:59:13.297308 8786 heartbeater.cc:464] Registering TS with master...
I20260812 06:59:13.297852 8786 heartbeater.cc:511] Master 127.19.52.126:45605 requested a full tablet report, sending...
I20260812 06:59:13.299391 8567 ts_manager.cc:194] Registered new tserver with Master: 7d62bb78c05d46b4a31f3925558aac7d (127.19.52.65:41247)
I20260812 06:59:13.300700 8567 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.19.52.65:38603
W20260812 06:59:13.408113 8790 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:59:13.408609 8790 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:59:13.408685 8790 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 06:59:13.416030 8790 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 06:59:13.416111 8790 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 06:59:13.416180 8790 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.52.66
I20260812 06:59:13.421715 8790 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.52.84:34387
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786517687384214-19665-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786517687384214-19665-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.52.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786517687384214-19665-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.19.52.66
--webserver_port=0
--tserver_master_addrs=127.19.52.126:45605
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8790
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.52.66
--log_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786517687384214-19665-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 06:15:54 UTC on ccycloud.quasar-rnfzce.root.comops.site
build id 4581989
I20260812 06:59:13.422966 8790 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:59:13.424407 8790 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 06:59:13.434548 8795 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:59:13.434567 8798 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:59:13.434500 8796 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:59:13.434732 8790 server_base.cc:1034] running on GCE node
I20260812 06:59:13.435235 8790 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:59:13.435938 8790 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:59:13.437127 8790 hybrid_clock.cc:648] HybridClock initialized: now 1786517953437101 us; error 32 us; skew 500 ppm
I20260812 06:59:13.439162 8790 webserver.cc:460] Webserver started at http://127.19.52.66:46737/ using document root <none> and password file <none>
I20260812 06:59:13.439800 8790 fs_manager.cc:359] Metadata directory not provided
I20260812 06:59:13.439903 8790 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:59:13.440140 8790 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 06:59:13.442373 8790 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786517687384214-19665-0/minicluster-data/ts-1/data/instance:
uuid: "0f7ab07b7bc44c6d9b95bf02b70485c3"
format_stamp: "Formatted at 2026-08-12 06:59:13 on dist-test-slave-65mx"
server_key: "a5e7125f14051f57b1fad2ac638e935e"
server_key_iv: "f2ab8cbe4dfe59e1c02b96db92ebc015"
server_key_version: "encryptionkey@0"
I20260812 06:59:13.442972 8790 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786517687384214-19665-0/minicluster-data/ts-1/wal/instance:
uuid: "0f7ab07b7bc44c6d9b95bf02b70485c3"
format_stamp: "Formatted at 2026-08-12 06:59:13 on dist-test-slave-65mx"
server_key: "a5e7125f14051f57b1fad2ac638e935e"
server_key_iv: "f2ab8cbe4dfe59e1c02b96db92ebc015"
server_key_version: "encryptionkey@0"
I20260812 06:59:13.446969 8790 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260812 06:59:13.449676 8805 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 06:59:13.451045 8790 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 06:59:13.451202 8790 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786517687384214-19665-0/minicluster-data/ts-1/data,/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786517687384214-19665-0/minicluster-data/ts-1/wal
uuid: "0f7ab07b7bc44c6d9b95bf02b70485c3"
format_stamp: "Formatted at 2026-08-12 06:59:13 on dist-test-slave-65mx"
server_key: "a5e7125f14051f57b1fad2ac638e935e"
server_key_iv: "f2ab8cbe4dfe59e1c02b96db92ebc015"
server_key_version: "encryptionkey@0"
I20260812 06:59:13.451323 8790 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786517687384214-19665-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786517687384214-19665-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786517687384214-19665-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:59:13.469920 8790 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 06:59:13.470675 8790 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 06:59:13.470891 8790 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:59:13.471539 8790 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 06:59:13.472626 8790 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 06:59:13.472697 8790 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 06:59:13.472770 8790 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 06:59:13.472816 8790 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 06:59:13.482457 8790 rpc_server.cc:307] RPC server started. Bound to: 127.19.52.66:33859
I20260812 06:59:13.482491 8942 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.52.66:33859 every 8 connection(s)
I20260812 06:59:13.483393 8790 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786517687384214-19665-0/minicluster-data/ts-1/data/info.pb
I20260812 06:59:13.489465 19665 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu as pid 8790
I20260812 06:59:13.489637 19665 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786517687384214-19665-0/minicluster-data/ts-1/wal/instance
I20260812 06:59:13.489972 19665 external_mini_cluster.cc:1428] Setting key 8fcd38753e2f357d9bd0f88649a4b974
I20260812 06:59:13.492369 19665 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu
/tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786517687384214-19665-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786517687384214-19665-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786517687384214-19665-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786517687384214-19665-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.52.67:0
--local_ip_for_outbound_sockets=127.19.52.67
--webserver_interface=127.19.52.67
--webserver_port=0
--tserver_master_addrs=127.19.52.126:45605
--builtin_ntp_servers=127.19.52.84:34387
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260812 06:59:13.493924 8614 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:59:13.485560 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.52.66:33681 (local address 127.19.52.126:45605)
0812 06:59:13.485704 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0812 06:59:13.485708 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 06:59:13.486881 (+ 1173us) server_negotiation.cc:407] Connection header received
0812 06:59:13.488409 (+ 1528us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:59:13.488412 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:59:13.488477 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:59:13.488647 (+ 170us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:59:13.489607 (+ 960us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:59:13.490271 (+ 664us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:59:13.491681 (+ 1410us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:59:13.491910 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:59:13.492026 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:59:13.493301 (+ 1275us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:59:13.493304 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:59:13.493306 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 06:59:13.493319 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:59:13.493339 (+ 20us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 06:59:13.493347 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:59:13.493427 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0812 06:59:13.493700 (+ 273us) server_negotiation.cc:299] Negotiation successful
0812 06:59:13.493774 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260812 06:59:13.494928 8945 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:59:13.485990 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.126:45605 (local address 127.19.52.66:33681)
0812 06:59:13.486716 (+ 726us) negotiation.cc:107] Waiting for socket to connect
0812 06:59:13.486760 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0812 06:59:13.488172 (+ 1412us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:59:13.488841 (+ 669us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:59:13.488848 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:59:13.488938 (+ 90us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:59:13.489452 (+ 514us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:59:13.489464 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:59:13.490426 (+ 962us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:59:13.490429 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:59:13.491545 (+ 1116us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:59:13.491555 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:59:13.492089 (+ 534us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:59:13.492094 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:59:13.492265 (+ 171us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:59:13.493001 (+ 736us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 06:59:13.493003 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:59:13.493048 (+ 45us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 06:59:13.493122 (+ 74us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 06:59:13.493429 (+ 307us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 06:59:13.493508 (+ 79us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 06:59:13.493520 (+ 12us) client_negotiation.cc:769] Sending connection context
0812 06:59:13.493693 (+ 173us) client_negotiation.cc:240] Negotiation successful
0812 06:59:13.494008 (+ 315us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":316,"thread_start_us":149,"threads_started":1}
I20260812 06:59:13.496193 8943 heartbeater.cc:347] Connected to a master server at 127.19.52.126:45605
I20260812 06:59:13.496465 8943 heartbeater.cc:464] Registering TS with master...
I20260812 06:59:13.497063 8943 heartbeater.cc:511] Master 127.19.52.126:45605 requested a full tablet report, sending...
I20260812 06:59:13.498253 8567 ts_manager.cc:194] Registered new tserver with Master: 0f7ab07b7bc44c6d9b95bf02b70485c3 (127.19.52.66:33859)
I20260812 06:59:13.499157 8567 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.19.52.66:33681
W20260812 06:59:13.608438 8946 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:59:13.608779 8946 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:59:13.608822 8946 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 06:59:13.613214 8946 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 06:59:13.613286 8946 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 06:59:13.613343 8946 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.52.67
I20260812 06:59:13.618237 8946 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.52.84:34387
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786517687384214-19665-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786517687384214-19665-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.52.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786517687384214-19665-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.19.52.67
--webserver_port=0
--tserver_master_addrs=127.19.52.126:45605
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8946
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.52.67
--log_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786517687384214-19665-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 06:15:54 UTC on ccycloud.quasar-rnfzce.root.comops.site
build id 4581989
I20260812 06:59:13.619401 8946 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:59:13.620844 8946 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 06:59:13.630833 8953 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:59:13.630820 8952 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:59:13.630820 8955 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:59:13.631047 8946 server_base.cc:1034] running on GCE node
I20260812 06:59:13.631636 8946 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:59:13.632264 8946 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:59:13.633482 8946 hybrid_clock.cc:648] HybridClock initialized: now 1786517953633473 us; error 49 us; skew 500 ppm
I20260812 06:59:13.635704 8946 webserver.cc:460] Webserver started at http://127.19.52.67:33221/ using document root <none> and password file <none>
I20260812 06:59:13.636430 8946 fs_manager.cc:359] Metadata directory not provided
I20260812 06:59:13.636541 8946 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:59:13.636831 8946 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 06:59:13.639192 8946 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786517687384214-19665-0/minicluster-data/ts-2/data/instance:
uuid: "872bd13b8db0402fa8263ed92ab07448"
format_stamp: "Formatted at 2026-08-12 06:59:13 on dist-test-slave-65mx"
server_key: "3e4019caa8d37e38bdc7d1887518d725"
server_key_iv: "c18749880e8ef2dbdbf6a93f632ae060"
server_key_version: "encryptionkey@0"
I20260812 06:59:13.639858 8946 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786517687384214-19665-0/minicluster-data/ts-2/wal/instance:
uuid: "872bd13b8db0402fa8263ed92ab07448"
format_stamp: "Formatted at 2026-08-12 06:59:13 on dist-test-slave-65mx"
server_key: "3e4019caa8d37e38bdc7d1887518d725"
server_key_iv: "c18749880e8ef2dbdbf6a93f632ae060"
server_key_version: "encryptionkey@0"
I20260812 06:59:13.644145 8946 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.004s
I20260812 06:59:13.646847 8963 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 06:59:13.648252 8946 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260812 06:59:13.648427 8946 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786517687384214-19665-0/minicluster-data/ts-2/data,/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786517687384214-19665-0/minicluster-data/ts-2/wal
uuid: "872bd13b8db0402fa8263ed92ab07448"
format_stamp: "Formatted at 2026-08-12 06:59:13 on dist-test-slave-65mx"
server_key: "3e4019caa8d37e38bdc7d1887518d725"
server_key_iv: "c18749880e8ef2dbdbf6a93f632ae060"
server_key_version: "encryptionkey@0"
I20260812 06:59:13.648597 8946 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786517687384214-19665-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786517687384214-19665-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786517687384214-19665-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:59:13.661569 8946 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 06:59:13.662325 8946 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 06:59:13.662559 8946 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:59:13.663189 8946 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 06:59:13.664222 8946 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 06:59:13.664294 8946 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 06:59:13.664363 8946 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 06:59:13.664412 8946 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 06:59:13.673483 8946 rpc_server.cc:307] RPC server started. Bound to: 127.19.52.67:33407
I20260812 06:59:13.673558 9092 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.52.67:33407 every 8 connection(s)
I20260812 06:59:13.674458 8946 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786517687384214-19665-0/minicluster-data/ts-2/data/info.pb
I20260812 06:59:13.680730 19665 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu as pid 8946
I20260812 06:59:13.680881 19665 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786517687384214-19665-0/minicluster-data/ts-2/wal/instance
I20260812 06:59:13.681202 19665 external_mini_cluster.cc:1428] Setting key 146a33e082f9541297edfba25f32fd0f
I20260812 06:59:13.685089 8614 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:59:13.676485 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.52.67:42037 (local address 127.19.52.126:45605)
0812 06:59:13.676674 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0812 06:59:13.676679 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 06:59:13.677868 (+ 1189us) server_negotiation.cc:407] Connection header received
0812 06:59:13.679448 (+ 1580us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:59:13.679451 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:59:13.679502 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:59:13.679590 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:59:13.680831 (+ 1241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:59:13.681491 (+ 660us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:59:13.683009 (+ 1518us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:59:13.683239 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:59:13.683350 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:59:13.684524 (+ 1174us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:59:13.684528 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:59:13.684530 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 06:59:13.684582 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:59:13.684604 (+ 22us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 06:59:13.684614 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:59:13.684698 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0812 06:59:13.684871 (+ 173us) server_negotiation.cc:299] Negotiation successful
0812 06:59:13.684934 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
I20260812 06:59:13.686025 9095 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:59:13.676959 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.126:45605 (local address 127.19.52.67:42037)
0812 06:59:13.677693 (+ 734us) negotiation.cc:107] Waiting for socket to connect
0812 06:59:13.677739 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0812 06:59:13.679225 (+ 1486us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:59:13.679780 (+ 555us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:59:13.679787 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:59:13.679879 (+ 92us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:59:13.680618 (+ 739us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:59:13.680643 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:59:13.681623 (+ 980us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:59:13.681626 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:59:13.682885 (+ 1259us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:59:13.682895 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:59:13.683403 (+ 508us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:59:13.683408 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:59:13.683587 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:59:13.684255 (+ 668us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 06:59:13.684257 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:59:13.684297 (+ 40us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 06:59:13.684363 (+ 66us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 06:59:13.684713 (+ 350us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 06:59:13.684725 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 06:59:13.684731 (+ 6us) client_negotiation.cc:769] Sending connection context
0812 06:59:13.684869 (+ 138us) client_negotiation.cc:240] Negotiation successful
0812 06:59:13.685185 (+ 316us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":312,"thread_start_us":124,"threads_started":1}
I20260812 06:59:13.687180 9093 heartbeater.cc:347] Connected to a master server at 127.19.52.126:45605
I20260812 06:59:13.687386 9093 heartbeater.cc:464] Registering TS with master...
I20260812 06:59:13.687881 9093 heartbeater.cc:511] Master 127.19.52.126:45605 requested a full tablet report, sending...
I20260812 06:59:13.688915 8567 ts_manager.cc:194] Registered new tserver with Master: 872bd13b8db0402fa8263ed92ab07448 (127.19.52.67:33407)
I20260812 06:59:13.689515 8567 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.19.52.67:42037
I20260812 06:59:13.695658 19665 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 06:59:13.702497 8614 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:59:13.697953 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49456 (local address 127.19.52.126:45605)
0812 06:59:13.698134 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0812 06:59:13.698139 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 06:59:13.698343 (+ 204us) server_negotiation.cc:407] Connection header received
0812 06:59:13.698499 (+ 156us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:59:13.698502 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:59:13.698551 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:59:13.698653 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:59:13.698981 (+ 328us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:59:13.699563 (+ 582us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:59:13.700507 (+ 944us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:59:13.700809 (+ 302us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:59:13.700930 (+ 121us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:59:13.701900 (+ 970us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:59:13.701907 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:59:13.701915 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 06:59:13.701929 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:59:13.701950 (+ 21us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 06:59:13.701961 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:59:13.702129 (+ 168us) server_negotiation.cc:1035] Waiting for connection context
0812 06:59:13.702270 (+ 141us) server_negotiation.cc:299] Negotiation successful
0812 06:59:13.702310 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260812 06:59:13.705637 8567 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:49456:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 06:59:13.708974 8567 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 06:59:13.724756 9106 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:59:13.718389 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.66:33859 (local address 127.0.0.1:50114)
0812 06:59:13.719255 (+ 866us) negotiation.cc:107] Waiting for socket to connect
0812 06:59:13.719742 (+ 487us) client_negotiation.cc:174] Beginning negotiation
0812 06:59:13.719820 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:59:13.720527 (+ 707us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:59:13.720530 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:59:13.720572 (+ 42us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 06:59:13.720669 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:59:13.720676 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:59:13.721870 (+ 1194us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:59:13.721874 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:59:13.723069 (+ 1195us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:59:13.723083 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:59:13.724128 (+ 1045us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:59:13.724131 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:59:13.724272 (+ 141us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:59:13.724317 (+ 45us) client_negotiation.cc:769] Sending connection context
0812 06:59:13.724438 (+ 121us) client_negotiation.cc:240] Negotiation successful
0812 06:59:13.724589 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":771,"mutex_wait_us":211,"thread_start_us":117,"threads_started":1}
I20260812 06:59:13.724817 9109 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:59:13.719154 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50114 (local address 127.19.52.66:33859)
0812 06:59:13.720094 (+ 940us) server_negotiation.cc:206] Beginning negotiation
0812 06:59:13.720106 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0812 06:59:13.720128 (+ 22us) server_negotiation.cc:407] Connection header received
0812 06:59:13.720198 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:59:13.720202 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:59:13.720377 (+ 175us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:59:13.720809 (+ 432us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 06:59:13.720976 (+ 167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:59:13.721713 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:59:13.723203 (+ 1490us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:59:13.723990 (+ 787us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:59:13.724113 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:59:13.724299 (+ 186us) server_negotiation.cc:1035] Waiting for connection context
0812 06:59:13.724486 (+ 187us) server_negotiation.cc:299] Negotiation successful
0812 06:59:13.724625 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":809,"thread_start_us":162,"threads_started":1}
I20260812 06:59:13.725039 9107 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:59:13.718571 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56798 (local address 127.19.52.67:33407)
0812 06:59:13.719082 (+ 511us) server_negotiation.cc:206] Beginning negotiation
0812 06:59:13.719096 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0812 06:59:13.719421 (+ 325us) server_negotiation.cc:407] Connection header received
0812 06:59:13.719763 (+ 342us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:59:13.719772 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:59:13.720001 (+ 229us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:59:13.720132 (+ 131us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 06:59:13.720725 (+ 593us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:59:13.721674 (+ 949us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:59:13.723159 (+ 1485us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:59:13.723990 (+ 831us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:59:13.724094 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:59:13.724299 (+ 205us) server_negotiation.cc:1035] Waiting for connection context
0812 06:59:13.724736 (+ 437us) server_negotiation.cc:299] Negotiation successful
0812 06:59:13.724857 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":390,"thread_start_us":161,"threads_started":1}
I20260812 06:59:13.725375 9105 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:59:13.718389 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.67:33407 (local address 127.0.0.1:56798)
0812 06:59:13.719255 (+ 866us) negotiation.cc:107] Waiting for socket to connect
0812 06:59:13.719313 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0812 06:59:13.719552 (+ 239us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:59:13.720353 (+ 801us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:59:13.720360 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:59:13.720382 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 06:59:13.720599 (+ 217us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:59:13.720606 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:59:13.721802 (+ 1196us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:59:13.721808 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:59:13.722979 (+ 1171us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:59:13.722993 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:59:13.724432 (+ 1439us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:59:13.724437 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:59:13.724537 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:59:13.724594 (+ 57us) client_negotiation.cc:769] Sending connection context
0812 06:59:13.725169 (+ 575us) client_negotiation.cc:240] Negotiation successful
0812 06:59:13.725250 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":697,"mutex_wait_us":59,"thread_start_us":114,"threads_started":1}
I20260812 06:59:13.727068 9108 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:59:13.718720 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.65:41247 (local address 127.0.0.1:56320)
0812 06:59:13.719662 (+ 942us) negotiation.cc:107] Waiting for socket to connect
0812 06:59:13.719678 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0812 06:59:13.719766 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:59:13.720600 (+ 834us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:59:13.720603 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:59:13.720625 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 06:59:13.721974 (+ 1349us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:59:13.721981 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:59:13.723991 (+ 2010us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:59:13.723996 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:59:13.725155 (+ 1159us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:59:13.725170 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:59:13.726681 (+ 1511us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:59:13.726685 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:59:13.726768 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:59:13.726788 (+ 20us) client_negotiation.cc:769] Sending connection context
0812 06:59:13.726851 (+ 63us) client_negotiation.cc:240] Negotiation successful
0812 06:59:13.726923 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":866,"mutex_wait_us":1185,"thread_start_us":93,"threads_started":1}
I20260812 06:59:13.727185 9110 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:59:13.719429 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56320 (local address 127.19.52.65:41247)
0812 06:59:13.720060 (+ 631us) server_negotiation.cc:206] Beginning negotiation
0812 06:59:13.720073 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0812 06:59:13.720097 (+ 24us) server_negotiation.cc:407] Connection header received
0812 06:59:13.720173 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:59:13.720178 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:59:13.720325 (+ 147us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:59:13.720631 (+ 306us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 06:59:13.722126 (+ 1495us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:59:13.723862 (+ 1736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:59:13.725309 (+ 1447us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:59:13.726548 (+ 1239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:59:13.726681 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:59:13.726878 (+ 197us) server_negotiation.cc:1035] Waiting for connection context
0812 06:59:13.726927 (+ 49us) server_negotiation.cc:299] Negotiation successful
0812 06:59:13.727016 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":512,"thread_start_us":151,"threads_started":1}
I20260812 06:59:13.728299 9021 tablet_service.cc:1467] Processing CreateTablet for tablet d4db8c5c694d4b1584bf94afba8bce89 (DEFAULT_TABLE table=test-table [id=0426bf6b378f4bb081e016b006292620]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 06:59:13.729370 9021 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d4db8c5c694d4b1584bf94afba8bce89. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 06:59:13.729843 8711 tablet_service.cc:1467] Processing CreateTablet for tablet d4db8c5c694d4b1584bf94afba8bce89 (DEFAULT_TABLE table=test-table [id=0426bf6b378f4bb081e016b006292620]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 06:59:13.729842 8862 tablet_service.cc:1467] Processing CreateTablet for tablet d4db8c5c694d4b1584bf94afba8bce89 (DEFAULT_TABLE table=test-table [id=0426bf6b378f4bb081e016b006292620]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 06:59:13.730878 8711 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d4db8c5c694d4b1584bf94afba8bce89. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 06:59:13.730878 8862 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d4db8c5c694d4b1584bf94afba8bce89. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 06:59:13.736188 9111 tablet_bootstrap.cc:492] T d4db8c5c694d4b1584bf94afba8bce89 P 872bd13b8db0402fa8263ed92ab07448: Bootstrap starting.
I20260812 06:59:13.738780 9111 tablet_bootstrap.cc:654] T d4db8c5c694d4b1584bf94afba8bce89 P 872bd13b8db0402fa8263ed92ab07448: Neither blocks nor log segments found. Creating new log.
I20260812 06:59:13.738848 9115 tablet_bootstrap.cc:492] T d4db8c5c694d4b1584bf94afba8bce89 P 0f7ab07b7bc44c6d9b95bf02b70485c3: Bootstrap starting.
I20260812 06:59:13.739631 9114 tablet_bootstrap.cc:492] T d4db8c5c694d4b1584bf94afba8bce89 P 7d62bb78c05d46b4a31f3925558aac7d: Bootstrap starting.
I20260812 06:59:13.739722 9111 log.cc:824] T d4db8c5c694d4b1584bf94afba8bce89 P 872bd13b8db0402fa8263ed92ab07448: Log is configured to *not* fsync() on all Append() calls
I20260812 06:59:13.741139 9115 tablet_bootstrap.cc:654] T d4db8c5c694d4b1584bf94afba8bce89 P 0f7ab07b7bc44c6d9b95bf02b70485c3: Neither blocks nor log segments found. Creating new log.
I20260812 06:59:13.741680 9114 tablet_bootstrap.cc:654] T d4db8c5c694d4b1584bf94afba8bce89 P 7d62bb78c05d46b4a31f3925558aac7d: Neither blocks nor log segments found. Creating new log.
I20260812 06:59:13.741999 9115 log.cc:824] T d4db8c5c694d4b1584bf94afba8bce89 P 0f7ab07b7bc44c6d9b95bf02b70485c3: Log is configured to *not* fsync() on all Append() calls
I20260812 06:59:13.742125 9111 tablet_bootstrap.cc:492] T d4db8c5c694d4b1584bf94afba8bce89 P 872bd13b8db0402fa8263ed92ab07448: No bootstrap required, opened a new log
I20260812 06:59:13.742331 9111 ts_tablet_manager.cc:1397] T d4db8c5c694d4b1584bf94afba8bce89 P 872bd13b8db0402fa8263ed92ab07448: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260812 06:59:13.742484 9114 log.cc:824] T d4db8c5c694d4b1584bf94afba8bce89 P 7d62bb78c05d46b4a31f3925558aac7d: Log is configured to *not* fsync() on all Append() calls
I20260812 06:59:13.744267 9115 tablet_bootstrap.cc:492] T d4db8c5c694d4b1584bf94afba8bce89 P 0f7ab07b7bc44c6d9b95bf02b70485c3: No bootstrap required, opened a new log
I20260812 06:59:13.744472 9115 ts_tablet_manager.cc:1397] T d4db8c5c694d4b1584bf94afba8bce89 P 0f7ab07b7bc44c6d9b95bf02b70485c3: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260812 06:59:13.744571 9114 tablet_bootstrap.cc:492] T d4db8c5c694d4b1584bf94afba8bce89 P 7d62bb78c05d46b4a31f3925558aac7d: No bootstrap required, opened a new log
I20260812 06:59:13.744755 9114 ts_tablet_manager.cc:1397] T d4db8c5c694d4b1584bf94afba8bce89 P 7d62bb78c05d46b4a31f3925558aac7d: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.003s
I20260812 06:59:13.746995 9111 raft_consensus.cc:348] T d4db8c5c694d4b1584bf94afba8bce89 P 872bd13b8db0402fa8263ed92ab07448 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "872bd13b8db0402fa8263ed92ab07448" member_type: VOTER last_known_addr { host: "127.19.52.67" port: 33407 } } peers { permanent_uuid: "0f7ab07b7bc44c6d9b95bf02b70485c3" member_type: VOTER last_known_addr { host: "127.19.52.66" port: 33859 } } peers { permanent_uuid: "7d62bb78c05d46b4a31f3925558aac7d" member_type: VOTER last_known_addr { host: "127.19.52.65" port: 41247 } }
I20260812 06:59:13.747315 9111 raft_consensus.cc:374] T d4db8c5c694d4b1584bf94afba8bce89 P 872bd13b8db0402fa8263ed92ab07448 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 06:59:13.747386 9111 raft_consensus.cc:729] T d4db8c5c694d4b1584bf94afba8bce89 P 872bd13b8db0402fa8263ed92ab07448 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 872bd13b8db0402fa8263ed92ab07448, State: Initialized, Role: FOLLOWER
I20260812 06:59:13.747881 9111 consensus_queue.cc:260] T d4db8c5c694d4b1584bf94afba8bce89 P 872bd13b8db0402fa8263ed92ab07448 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "872bd13b8db0402fa8263ed92ab07448" member_type: VOTER last_known_addr { host: "127.19.52.67" port: 33407 } } peers { permanent_uuid: "0f7ab07b7bc44c6d9b95bf02b70485c3" member_type: VOTER last_known_addr { host: "127.19.52.66" port: 33859 } } peers { permanent_uuid: "7d62bb78c05d46b4a31f3925558aac7d" member_type: VOTER last_known_addr { host: "127.19.52.65" port: 41247 } }
I20260812 06:59:13.748183 9114 raft_consensus.cc:348] T d4db8c5c694d4b1584bf94afba8bce89 P 7d62bb78c05d46b4a31f3925558aac7d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "872bd13b8db0402fa8263ed92ab07448" member_type: VOTER last_known_addr { host: "127.19.52.67" port: 33407 } } peers { permanent_uuid: "0f7ab07b7bc44c6d9b95bf02b70485c3" member_type: VOTER last_known_addr { host: "127.19.52.66" port: 33859 } } peers { permanent_uuid: "7d62bb78c05d46b4a31f3925558aac7d" member_type: VOTER last_known_addr { host: "127.19.52.65" port: 41247 } }
I20260812 06:59:13.748761 9093 heartbeater.cc:503] Master 127.19.52.126:45605 was elected leader, sending a full tablet report...
I20260812 06:59:13.748883 9114 raft_consensus.cc:374] T d4db8c5c694d4b1584bf94afba8bce89 P 7d62bb78c05d46b4a31f3925558aac7d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 06:59:13.748965 9114 raft_consensus.cc:729] T d4db8c5c694d4b1584bf94afba8bce89 P 7d62bb78c05d46b4a31f3925558aac7d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7d62bb78c05d46b4a31f3925558aac7d, State: Initialized, Role: FOLLOWER
I20260812 06:59:13.749007 9115 raft_consensus.cc:348] T d4db8c5c694d4b1584bf94afba8bce89 P 0f7ab07b7bc44c6d9b95bf02b70485c3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "872bd13b8db0402fa8263ed92ab07448" member_type: VOTER last_known_addr { host: "127.19.52.67" port: 33407 } } peers { permanent_uuid: "0f7ab07b7bc44c6d9b95bf02b70485c3" member_type: VOTER last_known_addr { host: "127.19.52.66" port: 33859 } } peers { permanent_uuid: "7d62bb78c05d46b4a31f3925558aac7d" member_type: VOTER last_known_addr { host: "127.19.52.65" port: 41247 } }
I20260812 06:59:13.749287 9115 raft_consensus.cc:374] T d4db8c5c694d4b1584bf94afba8bce89 P 0f7ab07b7bc44c6d9b95bf02b70485c3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 06:59:13.749342 9115 raft_consensus.cc:729] T d4db8c5c694d4b1584bf94afba8bce89 P 0f7ab07b7bc44c6d9b95bf02b70485c3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0f7ab07b7bc44c6d9b95bf02b70485c3, State: Initialized, Role: FOLLOWER
I20260812 06:59:13.749642 9114 consensus_queue.cc:260] T d4db8c5c694d4b1584bf94afba8bce89 P 7d62bb78c05d46b4a31f3925558aac7d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "872bd13b8db0402fa8263ed92ab07448" member_type: VOTER last_known_addr { host: "127.19.52.67" port: 33407 } } peers { permanent_uuid: "0f7ab07b7bc44c6d9b95bf02b70485c3" member_type: VOTER last_known_addr { host: "127.19.52.66" port: 33859 } } peers { permanent_uuid: "7d62bb78c05d46b4a31f3925558aac7d" member_type: VOTER last_known_addr { host: "127.19.52.65" port: 41247 } }
I20260812 06:59:13.750113 9111 ts_tablet_manager.cc:1428] T d4db8c5c694d4b1584bf94afba8bce89 P 872bd13b8db0402fa8263ed92ab07448: Time spent starting tablet: real 0.008s user 0.002s sys 0.006s
I20260812 06:59:13.750404 8786 heartbeater.cc:503] Master 127.19.52.126:45605 was elected leader, sending a full tablet report...
I20260812 06:59:13.750712 9114 ts_tablet_manager.cc:1428] T d4db8c5c694d4b1584bf94afba8bce89 P 7d62bb78c05d46b4a31f3925558aac7d: Time spent starting tablet: real 0.006s user 0.003s sys 0.001s
I20260812 06:59:13.750903 9115 consensus_queue.cc:260] T d4db8c5c694d4b1584bf94afba8bce89 P 0f7ab07b7bc44c6d9b95bf02b70485c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "872bd13b8db0402fa8263ed92ab07448" member_type: VOTER last_known_addr { host: "127.19.52.67" port: 33407 } } peers { permanent_uuid: "0f7ab07b7bc44c6d9b95bf02b70485c3" member_type: VOTER last_known_addr { host: "127.19.52.66" port: 33859 } } peers { permanent_uuid: "7d62bb78c05d46b4a31f3925558aac7d" member_type: VOTER last_known_addr { host: "127.19.52.65" port: 41247 } }
I20260812 06:59:13.751902 9115 ts_tablet_manager.cc:1428] T d4db8c5c694d4b1584bf94afba8bce89 P 0f7ab07b7bc44c6d9b95bf02b70485c3: Time spent starting tablet: real 0.007s user 0.007s sys 0.000s
I20260812 06:59:13.752012 8943 heartbeater.cc:503] Master 127.19.52.126:45605 was elected leader, sending a full tablet report...
I20260812 06:59:13.757745 9120 raft_consensus.cc:482] T d4db8c5c694d4b1584bf94afba8bce89 P 7d62bb78c05d46b4a31f3925558aac7d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 06:59:13.758021 9120 raft_consensus.cc:504] T d4db8c5c694d4b1584bf94afba8bce89 P 7d62bb78c05d46b4a31f3925558aac7d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "872bd13b8db0402fa8263ed92ab07448" member_type: VOTER last_known_addr { host: "127.19.52.67" port: 33407 } } peers { permanent_uuid: "0f7ab07b7bc44c6d9b95bf02b70485c3" member_type: VOTER last_known_addr { host: "127.19.52.66" port: 33859 } } peers { permanent_uuid: "7d62bb78c05d46b4a31f3925558aac7d" member_type: VOTER last_known_addr { host: "127.19.52.65" port: 41247 } }
I20260812 06:59:13.759155 9120 leader_election.cc:288] T d4db8c5c694d4b1584bf94afba8bce89 P 7d62bb78c05d46b4a31f3925558aac7d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 872bd13b8db0402fa8263ed92ab07448 (127.19.52.67:33407), 0f7ab07b7bc44c6d9b95bf02b70485c3 (127.19.52.66:33859)
I20260812 06:59:13.763737 8788 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:59:13.759415 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.66:33859 (local address 127.19.52.65:40411)
0812 06:59:13.759520 (+ 105us) negotiation.cc:107] Waiting for socket to connect
0812 06:59:13.759537 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0812 06:59:13.759629 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:59:13.760097 (+ 468us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:59:13.760100 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:59:13.760117 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 06:59:13.760228 (+ 111us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:59:13.760233 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:59:13.761350 (+ 1117us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:59:13.761353 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:59:13.762585 (+ 1232us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:59:13.762598 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:59:13.763366 (+ 768us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:59:13.763369 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:59:13.763468 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:59:13.763491 (+ 23us) client_negotiation.cc:769] Sending connection context
0812 06:59:13.763563 (+ 72us) client_negotiation.cc:240] Negotiation successful
0812 06:59:13.763624 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":27}
I20260812 06:59:13.763826 9109 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:59:13.759604 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.52.65:40411 (local address 127.19.52.66:33859)
0812 06:59:13.759742 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0812 06:59:13.759747 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 06:59:13.759765 (+ 18us) server_negotiation.cc:407] Connection header received
0812 06:59:13.759820 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:59:13.759824 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:59:13.759889 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:59:13.759976 (+ 87us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 06:59:13.760349 (+ 373us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:59:13.761221 (+ 872us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:59:13.762738 (+ 1517us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:59:13.763223 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:59:13.763353 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:59:13.763403 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0812 06:59:13.763602 (+ 199us) server_negotiation.cc:299] Negotiation successful
0812 06:59:13.763683 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260812 06:59:13.764631 9123 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:59:13.759415 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.67:33407 (local address 127.19.52.65:44203)
0812 06:59:13.759860 (+ 445us) negotiation.cc:107] Waiting for socket to connect
0812 06:59:13.759894 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0812 06:59:13.759978 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:59:13.760540 (+ 562us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:59:13.760546 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:59:13.760602 (+ 56us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 06:59:13.760764 (+ 162us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:59:13.760774 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:59:13.761836 (+ 1062us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:59:13.761840 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:59:13.762974 (+ 1134us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:59:13.762985 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:59:13.764172 (+ 1187us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:59:13.764176 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:59:13.764275 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:59:13.764309 (+ 34us) client_negotiation.cc:769] Sending connection context
0812 06:59:13.764383 (+ 74us) client_negotiation.cc:240] Negotiation successful
0812 06:59:13.764440 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":389,"mutex_wait_us":103,"thread_start_us":124,"threads_started":1}
I20260812 06:59:13.764648 9107 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:59:13.760022 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.52.65:44203 (local address 127.19.52.67:33407)
0812 06:59:13.760165 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0812 06:59:13.760170 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 06:59:13.760185 (+ 15us) server_negotiation.cc:407] Connection header received
0812 06:59:13.760238 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:59:13.760241 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:59:13.760288 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:59:13.760384 (+ 96us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 06:59:13.760905 (+ 521us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:59:13.761631 (+ 726us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:59:13.763113 (+ 1482us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:59:13.763718 (+ 605us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:59:13.763839 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:59:13.763894 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0812 06:59:13.764429 (+ 535us) server_negotiation.cc:299] Negotiation successful
0812 06:59:13.764510 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260812 06:59:13.764719 8887 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d4db8c5c694d4b1584bf94afba8bce89" candidate_uuid: "7d62bb78c05d46b4a31f3925558aac7d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0f7ab07b7bc44c6d9b95bf02b70485c3" is_pre_election: true
I20260812 06:59:13.765079 8887 raft_consensus.cc:2454] T d4db8c5c694d4b1584bf94afba8bce89 P 0f7ab07b7bc44c6d9b95bf02b70485c3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d62bb78c05d46b4a31f3925558aac7d in term 0.
I20260812 06:59:13.765424 9042 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d4db8c5c694d4b1584bf94afba8bce89" candidate_uuid: "7d62bb78c05d46b4a31f3925558aac7d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "872bd13b8db0402fa8263ed92ab07448" is_pre_election: true
I20260812 06:59:13.765736 9042 raft_consensus.cc:2454] T d4db8c5c694d4b1584bf94afba8bce89 P 872bd13b8db0402fa8263ed92ab07448 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d62bb78c05d46b4a31f3925558aac7d in term 0.
I20260812 06:59:13.765769 8656 leader_election.cc:302] T d4db8c5c694d4b1584bf94afba8bce89 P 7d62bb78c05d46b4a31f3925558aac7d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0f7ab07b7bc44c6d9b95bf02b70485c3, 7d62bb78c05d46b4a31f3925558aac7d; no voters:
I20260812 06:59:13.766144 9120 raft_consensus.cc:2781] T d4db8c5c694d4b1584bf94afba8bce89 P 7d62bb78c05d46b4a31f3925558aac7d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 06:59:13.766235 9120 raft_consensus.cc:482] T d4db8c5c694d4b1584bf94afba8bce89 P 7d62bb78c05d46b4a31f3925558aac7d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 06:59:13.766268 9120 raft_consensus.cc:3037] T d4db8c5c694d4b1584bf94afba8bce89 P 7d62bb78c05d46b4a31f3925558aac7d [term 0 FOLLOWER]: Advancing to term 1
I20260812 06:59:13.767428 9120 raft_consensus.cc:504] T d4db8c5c694d4b1584bf94afba8bce89 P 7d62bb78c05d46b4a31f3925558aac7d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "872bd13b8db0402fa8263ed92ab07448" member_type: VOTER last_known_addr { host: "127.19.52.67" port: 33407 } } peers { permanent_uuid: "0f7ab07b7bc44c6d9b95bf02b70485c3" member_type: VOTER last_known_addr { host: "127.19.52.66" port: 33859 } } peers { permanent_uuid: "7d62bb78c05d46b4a31f3925558aac7d" member_type: VOTER last_known_addr { host: "127.19.52.65" port: 41247 } }
I20260812 06:59:13.767805 9120 leader_election.cc:288] T d4db8c5c694d4b1584bf94afba8bce89 P 7d62bb78c05d46b4a31f3925558aac7d [CANDIDATE]: Term 1 election: Requested vote from peers 872bd13b8db0402fa8263ed92ab07448 (127.19.52.67:33407), 0f7ab07b7bc44c6d9b95bf02b70485c3 (127.19.52.66:33859)
I20260812 06:59:13.768244 8887 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d4db8c5c694d4b1584bf94afba8bce89" candidate_uuid: "7d62bb78c05d46b4a31f3925558aac7d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0f7ab07b7bc44c6d9b95bf02b70485c3"
I20260812 06:59:13.768359 9042 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d4db8c5c694d4b1584bf94afba8bce89" candidate_uuid: "7d62bb78c05d46b4a31f3925558aac7d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "872bd13b8db0402fa8263ed92ab07448"
I20260812 06:59:13.768365 8887 raft_consensus.cc:3037] T d4db8c5c694d4b1584bf94afba8bce89 P 0f7ab07b7bc44c6d9b95bf02b70485c3 [term 0 FOLLOWER]: Advancing to term 1
I20260812 06:59:13.768529 9042 raft_consensus.cc:3037] T d4db8c5c694d4b1584bf94afba8bce89 P 872bd13b8db0402fa8263ed92ab07448 [term 0 FOLLOWER]: Advancing to term 1
I20260812 06:59:13.769591 8887 raft_consensus.cc:2454] T d4db8c5c694d4b1584bf94afba8bce89 P 0f7ab07b7bc44c6d9b95bf02b70485c3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7d62bb78c05d46b4a31f3925558aac7d in term 1.
I20260812 06:59:13.769629 9042 raft_consensus.cc:2454] T d4db8c5c694d4b1584bf94afba8bce89 P 872bd13b8db0402fa8263ed92ab07448 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7d62bb78c05d46b4a31f3925558aac7d in term 1.
I20260812 06:59:13.770131 8657 leader_election.cc:302] T d4db8c5c694d4b1584bf94afba8bce89 P 7d62bb78c05d46b4a31f3925558aac7d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d62bb78c05d46b4a31f3925558aac7d, 872bd13b8db0402fa8263ed92ab07448; no voters:
I20260812 06:59:13.770380 9120 raft_consensus.cc:2781] T d4db8c5c694d4b1584bf94afba8bce89 P 7d62bb78c05d46b4a31f3925558aac7d [term 1 FOLLOWER]: Leader election won for term 1
I20260812 06:59:13.770668 9120 raft_consensus.cc:686] T d4db8c5c694d4b1584bf94afba8bce89 P 7d62bb78c05d46b4a31f3925558aac7d [term 1 LEADER]: Becoming Leader. State: Replica: 7d62bb78c05d46b4a31f3925558aac7d, State: Running, Role: LEADER
I20260812 06:59:13.771051 9120 consensus_queue.cc:237] T d4db8c5c694d4b1584bf94afba8bce89 P 7d62bb78c05d46b4a31f3925558aac7d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "872bd13b8db0402fa8263ed92ab07448" member_type: VOTER last_known_addr { host: "127.19.52.67" port: 33407 } } peers { permanent_uuid: "0f7ab07b7bc44c6d9b95bf02b70485c3" member_type: VOTER last_known_addr { host: "127.19.52.66" port: 33859 } } peers { permanent_uuid: "7d62bb78c05d46b4a31f3925558aac7d" member_type: VOTER last_known_addr { host: "127.19.52.65" port: 41247 } }
I20260812 06:59:13.774401 8565 catalog_manager.cc:5495] T d4db8c5c694d4b1584bf94afba8bce89 P 7d62bb78c05d46b4a31f3925558aac7d reported cstate change: term changed from 0 to 1, leader changed from <none> to 7d62bb78c05d46b4a31f3925558aac7d (127.19.52.65). New cstate: current_term: 1 leader_uuid: "7d62bb78c05d46b4a31f3925558aac7d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "872bd13b8db0402fa8263ed92ab07448" member_type: VOTER last_known_addr { host: "127.19.52.67" port: 33407 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0f7ab07b7bc44c6d9b95bf02b70485c3" member_type: VOTER last_known_addr { host: "127.19.52.66" port: 33859 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7d62bb78c05d46b4a31f3925558aac7d" member_type: VOTER last_known_addr { host: "127.19.52.65" port: 41247 } health_report { overall_health: HEALTHY } } }
W20260812 06:59:13.785177 8787 tablet.cc:2369] T d4db8c5c694d4b1584bf94afba8bce89 P 7d62bb78c05d46b4a31f3925558aac7d: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 06:59:13.793399 9110 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:59:13.789177 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56326 (local address 127.19.52.65:41247)
0812 06:59:13.789330 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0812 06:59:13.789334 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 06:59:13.789350 (+ 16us) server_negotiation.cc:407] Connection header received
0812 06:59:13.789535 (+ 185us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:59:13.789540 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:59:13.789618 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:59:13.789771 (+ 153us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 06:59:13.790140 (+ 369us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:59:13.790845 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:59:13.791730 (+ 885us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:59:13.791952 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:59:13.792066 (+ 114us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:59:13.792392 (+ 326us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 06:59:13.792882 (+ 490us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 06:59:13.793084 (+ 202us) server_negotiation.cc:1035] Waiting for connection context
0812 06:59:13.793192 (+ 108us) server_negotiation.cc:299] Negotiation successful
0812 06:59:13.793242 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260812 06:59:13.799664 8887 raft_consensus.cc:1261] T d4db8c5c694d4b1584bf94afba8bce89 P 0f7ab07b7bc44c6d9b95bf02b70485c3 [term 1 FOLLOWER]: Refusing update from remote peer 7d62bb78c05d46b4a31f3925558aac7d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 06:59:13.799664 9042 raft_consensus.cc:1261] T d4db8c5c694d4b1584bf94afba8bce89 P 872bd13b8db0402fa8263ed92ab07448 [term 1 FOLLOWER]: Refusing update from remote peer 7d62bb78c05d46b4a31f3925558aac7d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 06:59:13.800462 9124 consensus_queue.cc:1035] T d4db8c5c694d4b1584bf94afba8bce89 P 7d62bb78c05d46b4a31f3925558aac7d [LEADER]: Connected to new peer: Peer: permanent_uuid: "872bd13b8db0402fa8263ed92ab07448" member_type: VOTER last_known_addr { host: "127.19.52.67" port: 33407 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 06:59:13.800719 9120 consensus_queue.cc:1035] T d4db8c5c694d4b1584bf94afba8bce89 P 7d62bb78c05d46b4a31f3925558aac7d [LEADER]: Connected to new peer: Peer: permanent_uuid: "0f7ab07b7bc44c6d9b95bf02b70485c3" member_type: VOTER last_known_addr { host: "127.19.52.66" port: 33859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 06:59:13.807909 9128 mvcc.cc:205] Tried to move back new op lower bound from 7317577538756116480 to 7317577538649317376. Current Snapshot: MvccSnapshot[applied={T|T < 7317577538756116480}]
I20260812 06:59:13.808813 9129 mvcc.cc:205] Tried to move back new op lower bound from 7317577538756116480 to 7317577538649317376. Current Snapshot: MvccSnapshot[applied={T|T < 7317577538756116480}]
I20260812 06:59:13.819087 8567 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:49456:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 06:59:13.819257 8567 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:49456:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 06:59:13.822090 8567 catalog_manager.cc:5782] T 00000000000000000000000000000000 P d91f95e589d044688c362db5a0edeb96: Sending DeleteTablet for 3 replicas of tablet d4db8c5c694d4b1584bf94afba8bce89
I20260812 06:59:13.823124 8711 tablet_service.cc:1514] Processing DeleteTablet for tablet d4db8c5c694d4b1584bf94afba8bce89 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 06:59:13 UTC) from {username='slave'} at 127.0.0.1:56320
I20260812 06:59:13.823123 8862 tablet_service.cc:1514] Processing DeleteTablet for tablet d4db8c5c694d4b1584bf94afba8bce89 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 06:59:13 UTC) from {username='slave'} at 127.0.0.1:50114
I20260812 06:59:13.823140 9021 tablet_service.cc:1514] Processing DeleteTablet for tablet d4db8c5c694d4b1584bf94afba8bce89 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 06:59:13 UTC) from {username='slave'} at 127.0.0.1:56798
I20260812 06:59:13.823711 9137 tablet_replica.cc:330] T d4db8c5c694d4b1584bf94afba8bce89 P 872bd13b8db0402fa8263ed92ab07448: stopping tablet replica
I20260812 06:59:13.824002 9139 tablet_replica.cc:330] T d4db8c5c694d4b1584bf94afba8bce89 P 7d62bb78c05d46b4a31f3925558aac7d: stopping tablet replica
I20260812 06:59:13.824039 19665 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu with pid 8639
I20260812 06:59:13.824076 9137 raft_consensus.cc:2229] T d4db8c5c694d4b1584bf94afba8bce89 P 872bd13b8db0402fa8263ed92ab07448 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 06:59:13.824292 9139 raft_consensus.cc:2229] T d4db8c5c694d4b1584bf94afba8bce89 P 7d62bb78c05d46b4a31f3925558aac7d [term 1 LEADER]: Raft consensus shutting down.
I20260812 06:59:13.824424 9137 raft_consensus.cc:2258] T d4db8c5c694d4b1584bf94afba8bce89 P 872bd13b8db0402fa8263ed92ab07448 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 06:59:13.824928 9138 tablet_replica.cc:330] T d4db8c5c694d4b1584bf94afba8bce89 P 0f7ab07b7bc44c6d9b95bf02b70485c3: stopping tablet replica
I20260812 06:59:13.825232 9138 raft_consensus.cc:2229] T d4db8c5c694d4b1584bf94afba8bce89 P 0f7ab07b7bc44c6d9b95bf02b70485c3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 06:59:13.825558 9138 raft_consensus.cc:2258] T d4db8c5c694d4b1584bf94afba8bce89 P 0f7ab07b7bc44c6d9b95bf02b70485c3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 06:59:13.828142 9137 ts_tablet_manager.cc:1905] T d4db8c5c694d4b1584bf94afba8bce89 P 872bd13b8db0402fa8263ed92ab07448: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 06:59:13.828377 9138 ts_tablet_manager.cc:1905] T d4db8c5c694d4b1584bf94afba8bce89 P 0f7ab07b7bc44c6d9b95bf02b70485c3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 06:59:13.830909 9137 ts_tablet_manager.cc:1918] T d4db8c5c694d4b1584bf94afba8bce89 P 872bd13b8db0402fa8263ed92ab07448: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 06:59:13.831037 9137 log.cc:1196] T d4db8c5c694d4b1584bf94afba8bce89 P 872bd13b8db0402fa8263ed92ab07448: Deleting WAL directory at /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786517687384214-19665-0/minicluster-data/ts-2/wal/wals/d4db8c5c694d4b1584bf94afba8bce89
I20260812 06:59:13.831571 9137 ts_tablet_manager.cc:1939] T d4db8c5c694d4b1584bf94afba8bce89 P 872bd13b8db0402fa8263ed92ab07448: Deleting consensus metadata
I20260812 06:59:13.831777 9138 ts_tablet_manager.cc:1918] T d4db8c5c694d4b1584bf94afba8bce89 P 0f7ab07b7bc44c6d9b95bf02b70485c3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 06:59:13.831892 9138 log.cc:1196] T d4db8c5c694d4b1584bf94afba8bce89 P 0f7ab07b7bc44c6d9b95bf02b70485c3: Deleting WAL directory at /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786517687384214-19665-0/minicluster-data/ts-1/wal/wals/d4db8c5c694d4b1584bf94afba8bce89
I20260812 06:59:13.832226 9138 ts_tablet_manager.cc:1939] T d4db8c5c694d4b1584bf94afba8bce89 P 0f7ab07b7bc44c6d9b95bf02b70485c3: Deleting consensus metadata
I20260812 06:59:13.833029 8550 catalog_manager.cc:4841] TS 872bd13b8db0402fa8263ed92ab07448 (127.19.52.67:33407): tablet d4db8c5c694d4b1584bf94afba8bce89 (table test-table [id=0426bf6b378f4bb081e016b006292620]) successfully deleted
I20260812 06:59:13.833509 8549 catalog_manager.cc:4841] TS 0f7ab07b7bc44c6d9b95bf02b70485c3 (127.19.52.66:33859): tablet d4db8c5c694d4b1584bf94afba8bce89 (table test-table [id=0426bf6b378f4bb081e016b006292620]) successfully deleted
W20260812 06:59:13.835913 8548 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.19.52.65:41247 (error 108)
I20260812 06:59:13.836858 19665 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu with pid 8790
I20260812 06:59:13.837178 9108 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:59:13.836832 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.65:41247 (local address 127.0.0.1:56342)
0812 06:59:13.836986 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0812 06:59:13.837075 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.52.65:41247: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260812 06:59:13.840106 8548 catalog_manager.cc:4584] TS 7d62bb78c05d46b4a31f3925558aac7d (127.19.52.65:41247): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet d4db8c5c694d4b1584bf94afba8bce89: Network error: Client connection negotiation failed: client connection to 127.19.52.65:41247: connect: Connection refused (error 111)
I20260812 06:59:13.847157 19665 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu with pid 8946
I20260812 06:59:13.855464 19665 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu with pid 8530
2026-08-12T06:59:13Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/3 (1101 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/4
Loading random data
Initializing database '/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786517687384214-19665-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 06:59:13 dist-test-slave-65mx krb5kdc[9144](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 06:59:13 dist-test-slave-65mx krb5kdc[9144](info): set up 2 sockets
Aug 12 06:59:13 dist-test-slave-65mx krb5kdc[9144](info): commencing operation
krb5kdc: starting...
W20260812 06:59:14.974391 19665 mini_kdc.cc:121] Time spent starting KDC: real 1.094s user 0.004s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786517687384214-19665-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786517687384214-19665-0/krb5kdc/test-user.keytab.
Aug 12 06:59:15 dist-test-slave-65mx krb5kdc[9144](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517955, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T06:59:15Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T06:59:15Z Disabled control of system clock
WARNING: no policy specified for kudu/127.19.52.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.52.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.52.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786517687384214-19665-0/krb5kdc/kudu_127.19.52.126.keytab.
Entry for principal kudu/127.19.52.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786517687384214-19665-0/krb5kdc/kudu_127.19.52.126.keytab.
I20260812 06:59:15.116389 19665 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu
/tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786517687384214-19665-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786517687384214-19665-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786517687384214-19665-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786517687384214-19665-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.52.126:35849
--webserver_interface=127.19.52.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786517687384214-19665-0/krb5kdc/kudu_127.19.52.126.keytab
--principal=kudu/127.19.52.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.52.84:36075
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.19.52.126:35849
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786517687384214-19665-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786517687384214-19665-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786517687384214-19665-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 06:59:15.241194 9158 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:59:15.241561 9158 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:59:15.241649 9158 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 06:59:15.245931 9158 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 06:59:15.246034 9158 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 06:59:15.246076 9158 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 06:59:15.246145 9158 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 06:59:15.246201 9158 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 06:59:15.251965 9158 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.52.84:36075
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786517687384214-19665-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786517687384214-19665-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.19.52.126:35849
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786517687384214-19665-0/krb5kdc/kudu_127.19.52.126.keytab
--principal=kudu/127.19.52.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.52.126:35849
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786517687384214-19665-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.19.52.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9158
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786517687384214-19665-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 06:15:54 UTC on ccycloud.quasar-rnfzce.root.comops.site
build id 4581989
I20260812 06:59:15.253340 9158 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:59:15.254478 9158 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 06:59:15.263618 9167 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:59:15.263635 9168 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:59:15.263634 9170 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:59:15.263813 9158 server_base.cc:1034] running on GCE node
I20260812 06:59:15.264612 9158 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:59:15.265843 9158 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:59:15.267051 9158 hybrid_clock.cc:648] HybridClock initialized: now 1786517955267030 us; error 34 us; skew 500 ppm
Aug 12 06:59:15 dist-test-slave-65mx krb5kdc[9144](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517955, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.52.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 06:59:15.270507 9158 init.cc:377] Logged in from keytab as kudu/127.19.52.126@KRBTEST.COM (short username kudu)
I20260812 06:59:15.271577 9158 webserver.cc:460] Webserver started at http://127.19.52.126:41449/ using document root <none> and password file <none>
I20260812 06:59:15.272200 9158 fs_manager.cc:359] Metadata directory not provided
I20260812 06:59:15.272269 9158 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:59:15.272506 9158 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 06:59:15.274941 9158 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786517687384214-19665-0/minicluster-data/master-0/data/instance:
uuid: "101f82f261d942bc89027bc200f1fab2"
format_stamp: "Formatted at 2026-08-12 06:59:15 on dist-test-slave-65mx"
server_key: "4f6209d469388f892bd67be3fbb5aaaa"
server_key_iv: "8fde76d550809ba98555363d6b1b053b"
server_key_version: "encryptionkey@0"
I20260812 06:59:15.275594 9158 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786517687384214-19665-0/minicluster-data/master-0/wal/instance:
uuid: "101f82f261d942bc89027bc200f1fab2"
format_stamp: "Formatted at 2026-08-12 06:59:15 on dist-test-slave-65mx"
server_key: "4f6209d469388f892bd67be3fbb5aaaa"
server_key_iv: "8fde76d550809ba98555363d6b1b053b"
server_key_version: "encryptionkey@0"
I20260812 06:59:15.279866 9158 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.006s
I20260812 06:59:15.282826 9181 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 06:59:15.284490 9158 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260812 06:59:15.284693 9158 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786517687384214-19665-0/minicluster-data/master-0/data,/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786517687384214-19665-0/minicluster-data/master-0/wal
uuid: "101f82f261d942bc89027bc200f1fab2"
format_stamp: "Formatted at 2026-08-12 06:59:15 on dist-test-slave-65mx"
server_key: "4f6209d469388f892bd67be3fbb5aaaa"
server_key_iv: "8fde76d550809ba98555363d6b1b053b"
server_key_version: "encryptionkey@0"
I20260812 06:59:15.284858 9158 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786517687384214-19665-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786517687384214-19665-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786517687384214-19665-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:59:15.303220 9158 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 06:59:15.306576 9158 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 06:59:15.306802 9158 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:59:15.314533 9158 rpc_server.cc:307] RPC server started. Bound to: 127.19.52.126:35849
I20260812 06:59:15.314546 9243 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.52.126:35849 every 8 connection(s)
I20260812 06:59:15.315666 9158 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786517687384214-19665-0/minicluster-data/master-0/data/info.pb
I20260812 06:59:15.318715 9244 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 06:59:15.324954 9244 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 101f82f261d942bc89027bc200f1fab2: Bootstrap starting.
I20260812 06:59:15.325407 19665 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu as pid 9158
I20260812 06:59:15.325598 19665 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786517687384214-19665-0/minicluster-data/master-0/wal/instance
I20260812 06:59:15.325891 19665 external_mini_cluster.cc:1428] Setting key 654823fe4312a5a301fc51c9d19f8080
I20260812 06:59:15.328059 9244 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 101f82f261d942bc89027bc200f1fab2: Neither blocks nor log segments found. Creating new log.
I20260812 06:59:15.328931 9244 log.cc:824] T 00000000000000000000000000000000 P 101f82f261d942bc89027bc200f1fab2: Log is configured to *not* fsync() on all Append() calls
I20260812 06:59:15.331256 9244 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 101f82f261d942bc89027bc200f1fab2: No bootstrap required, opened a new log
Aug 12 06:59:15 dist-test-slave-65mx krb5kdc[9144](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517955, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.19.52.126@KRBTEST.COM
I20260812 06:59:15.335781 9244 raft_consensus.cc:348] T 00000000000000000000000000000000 P 101f82f261d942bc89027bc200f1fab2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "101f82f261d942bc89027bc200f1fab2" member_type: VOTER last_known_addr { host: "127.19.52.126" port: 35849 } }
I20260812 06:59:15.336144 9244 raft_consensus.cc:374] T 00000000000000000000000000000000 P 101f82f261d942bc89027bc200f1fab2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 06:59:15.336215 9244 raft_consensus.cc:729] T 00000000000000000000000000000000 P 101f82f261d942bc89027bc200f1fab2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 101f82f261d942bc89027bc200f1fab2, State: Initialized, Role: FOLLOWER
I20260812 06:59:15.336884 9244 consensus_queue.cc:260] T 00000000000000000000000000000000 P 101f82f261d942bc89027bc200f1fab2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "101f82f261d942bc89027bc200f1fab2" member_type: VOTER last_known_addr { host: "127.19.52.126" port: 35849 } }
I20260812 06:59:15.337121 9244 raft_consensus.cc:388] T 00000000000000000000000000000000 P 101f82f261d942bc89027bc200f1fab2 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 06:59:15.337216 9244 raft_consensus.cc:482] T 00000000000000000000000000000000 P 101f82f261d942bc89027bc200f1fab2 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 06:59:15.337333 9244 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 101f82f261d942bc89027bc200f1fab2 [term 0 FOLLOWER]: Advancing to term 1
I20260812 06:59:15.338845 9244 raft_consensus.cc:504] T 00000000000000000000000000000000 P 101f82f261d942bc89027bc200f1fab2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "101f82f261d942bc89027bc200f1fab2" member_type: VOTER last_known_addr { host: "127.19.52.126" port: 35849 } }
I20260812 06:59:15.339298 9244 leader_election.cc:302] T 00000000000000000000000000000000 P 101f82f261d942bc89027bc200f1fab2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 101f82f261d942bc89027bc200f1fab2; no voters:
I20260812 06:59:15.339740 9244 leader_election.cc:288] T 00000000000000000000000000000000 P 101f82f261d942bc89027bc200f1fab2 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 06:59:15.339906 9249 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 101f82f261d942bc89027bc200f1fab2 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 06:59:15.340214 9249 raft_consensus.cc:686] T 00000000000000000000000000000000 P 101f82f261d942bc89027bc200f1fab2 [term 1 LEADER]: Becoming Leader. State: Replica: 101f82f261d942bc89027bc200f1fab2, State: Running, Role: LEADER
I20260812 06:59:15.340746 9249 consensus_queue.cc:237] T 00000000000000000000000000000000 P 101f82f261d942bc89027bc200f1fab2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "101f82f261d942bc89027bc200f1fab2" member_type: VOTER last_known_addr { host: "127.19.52.126" port: 35849 } }
I20260812 06:59:15.341329 9244 sys_catalog.cc:564] T 00000000000000000000000000000000 P 101f82f261d942bc89027bc200f1fab2 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 06:59:15.343148 9247 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:59:15.327541 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57552 (local address 127.19.52.126:35849)
0812 06:59:15.328007 (+ 466us) server_negotiation.cc:206] Beginning negotiation
0812 06:59:15.328024 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0812 06:59:15.328062 (+ 38us) server_negotiation.cc:407] Connection header received
0812 06:59:15.328863 (+ 801us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:59:15.328883 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:59:15.329304 (+ 421us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:59:15.329671 (+ 367us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:59:15.330540 (+ 869us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:59:15.331616 (+ 1076us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:59:15.332507 (+ 891us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:59:15.333082 (+ 575us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:59:15.333262 (+ 180us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:59:15.336076 (+ 2814us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:59:15.336084 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:59:15.336106 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:59:15.336148 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:59:15.338638 (+ 2490us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:59:15.339242 (+ 604us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:59:15.339249 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:59:15.339256 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:59:15.339419 (+ 163us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:59:15.339901 (+ 482us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:59:15.339904 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:59:15.339906 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:59:15.340489 (+ 583us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:59:15.341391 (+ 902us) server_negotiation.cc:1035] Waiting for connection context
0812 06:59:15.341556 (+ 165us) server_negotiation.cc:299] Negotiation successful
0812 06:59:15.341944 (+ 388us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":300,"thread_start_us":152,"threads_started":1}
I20260812 06:59:15.343263 9251 sys_catalog.cc:455] T 00000000000000000000000000000000 P 101f82f261d942bc89027bc200f1fab2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "101f82f261d942bc89027bc200f1fab2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "101f82f261d942bc89027bc200f1fab2" member_type: VOTER last_known_addr { host: "127.19.52.126" port: 35849 } } }
I20260812 06:59:15.343348 9251 sys_catalog.cc:458] T 00000000000000000000000000000000 P 101f82f261d942bc89027bc200f1fab2 [sys.catalog]: This master's current role is: LEADER
I20260812 06:59:15.343103 9252 sys_catalog.cc:455] T 00000000000000000000000000000000 P 101f82f261d942bc89027bc200f1fab2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 101f82f261d942bc89027bc200f1fab2. Latest consensus state: current_term: 1 leader_uuid: "101f82f261d942bc89027bc200f1fab2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "101f82f261d942bc89027bc200f1fab2" member_type: VOTER last_known_addr { host: "127.19.52.126" port: 35849 } } }
I20260812 06:59:15.344264 9252 sys_catalog.cc:458] T 00000000000000000000000000000000 P 101f82f261d942bc89027bc200f1fab2 [sys.catalog]: This master's current role is: LEADER
W20260812 06:59:15.347023 9268 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 101f82f261d942bc89027bc200f1fab2: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 06:59:15.347126 9268 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 06:59:15.347193 9267 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 06:59:15.347988 9267 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 06:59:15.355424 9267 catalog_manager.cc:1347] Generated new cluster ID: 369645b570d74c3d88d489196a8df18b
I20260812 06:59:15.355556 9267 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 06:59:15.376298 9267 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 06:59:15.377511 9267 catalog_manager.cc:1504] Loading token signing keys...
I20260812 06:59:15.387638 9267 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 101f82f261d942bc89027bc200f1fab2: Generated new TSK 0
I20260812 06:59:15.388496 9267 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.19.52.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.52.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.52.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786517687384214-19665-0/krb5kdc/kudu_127.19.52.65.keytab.
Entry for principal kudu/127.19.52.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786517687384214-19665-0/krb5kdc/kudu_127.19.52.65.keytab.
I20260812 06:59:15.426306 19665 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu
/tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786517687384214-19665-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786517687384214-19665-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786517687384214-19665-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786517687384214-19665-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.52.65:0
--local_ip_for_outbound_sockets=127.19.52.65
--webserver_interface=127.19.52.65
--webserver_port=0
--tserver_master_addrs=127.19.52.126:35849
--keytab_file=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786517687384214-19665-0/krb5kdc/kudu_127.19.52.65.keytab
--principal=kudu/127.19.52.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.52.84:36075
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786517687384214-19665-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786517687384214-19665-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786517687384214-19665-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 06:59:15.568804 9277 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:59:15.569195 9277 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:59:15.569247 9277 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 06:59:15.573585 9277 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 06:59:15.573673 9277 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 06:59:15.573735 9277 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.52.65
I20260812 06:59:15.579419 9277 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.52.84:36075
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786517687384214-19665-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786517687384214-19665-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786517687384214-19665-0/krb5kdc/kudu_127.19.52.65.keytab
--principal=kudu/127.19.52.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.52.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786517687384214-19665-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.19.52.65
--webserver_port=0
--tserver_master_addrs=127.19.52.126:35849
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9277
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.52.65
--log_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786517687384214-19665-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 06:15:54 UTC on ccycloud.quasar-rnfzce.root.comops.site
build id 4581989
I20260812 06:59:15.580729 9277 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:59:15.581825 9277 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 06:59:15.592267 9284 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:59:15.592283 9285 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:59:15.592312 9287 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:59:15.592581 9277 server_base.cc:1034] running on GCE node
I20260812 06:59:15.593165 9277 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:59:15.593935 9277 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:59:15.595214 9277 hybrid_clock.cc:648] HybridClock initialized: now 1786517955595188 us; error 36 us; skew 500 ppm
Aug 12 06:59:15 dist-test-slave-65mx krb5kdc[9144](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517955, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.52.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 06:59:15.598718 9277 init.cc:377] Logged in from keytab as kudu/127.19.52.65@KRBTEST.COM (short username kudu)
I20260812 06:59:15.599689 9277 webserver.cc:460] Webserver started at http://127.19.52.65:38771/ using document root <none> and password file <none>
I20260812 06:59:15.600330 9277 fs_manager.cc:359] Metadata directory not provided
I20260812 06:59:15.600402 9277 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:59:15.600696 9277 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 06:59:15.603204 9277 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786517687384214-19665-0/minicluster-data/ts-0/data/instance:
uuid: "669c52c47fb144edbe5fb712a33cbcf0"
format_stamp: "Formatted at 2026-08-12 06:59:15 on dist-test-slave-65mx"
server_key: "36008d36ec30ea1fc373edaffe6d8cdb"
server_key_iv: "daa8c17bd715fbb4bdd1ba7f49b088c6"
server_key_version: "encryptionkey@0"
I20260812 06:59:15.603861 9277 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786517687384214-19665-0/minicluster-data/ts-0/wal/instance:
uuid: "669c52c47fb144edbe5fb712a33cbcf0"
format_stamp: "Formatted at 2026-08-12 06:59:15 on dist-test-slave-65mx"
server_key: "36008d36ec30ea1fc373edaffe6d8cdb"
server_key_iv: "daa8c17bd715fbb4bdd1ba7f49b088c6"
server_key_version: "encryptionkey@0"
I20260812 06:59:15.608168 9277 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.002s
I20260812 06:59:15.611321 9296 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 06:59:15.612839 9277 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 06:59:15.613011 9277 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786517687384214-19665-0/minicluster-data/ts-0/data,/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786517687384214-19665-0/minicluster-data/ts-0/wal
uuid: "669c52c47fb144edbe5fb712a33cbcf0"
format_stamp: "Formatted at 2026-08-12 06:59:15 on dist-test-slave-65mx"
server_key: "36008d36ec30ea1fc373edaffe6d8cdb"
server_key_iv: "daa8c17bd715fbb4bdd1ba7f49b088c6"
server_key_version: "encryptionkey@0"
I20260812 06:59:15.613150 9277 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786517687384214-19665-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786517687384214-19665-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786517687384214-19665-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:59:15.652964 9277 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 06:59:15.657074 9277 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 06:59:15.657316 9277 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:59:15.657971 9277 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 06:59:15.658892 9277 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 06:59:15.658967 9277 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 06:59:15.659054 9277 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 06:59:15.659106 9277 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 06:59:15.669309 9277 rpc_server.cc:307] RPC server started. Bound to: 127.19.52.65:45275
I20260812 06:59:15.669337 9428 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.52.65:45275 every 8 connection(s)
I20260812 06:59:15.670329 9277 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786517687384214-19665-0/minicluster-data/ts-0/data/info.pb
I20260812 06:59:15.676803 19665 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu as pid 9277
I20260812 06:59:15.676966 19665 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786517687384214-19665-0/minicluster-data/ts-0/wal/instance
I20260812 06:59:15.677295 19665 external_mini_cluster.cc:1428] Setting key 1c2aa71cc61ac035e959c785d447a6f1
Aug 12 06:59:15 dist-test-slave-65mx krb5kdc[9144](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517955, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.52.65@KRBTEST.COM for kudu/127.19.52.126@KRBTEST.COM
I20260812 06:59:15.687371 9247 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:59:15.672397 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.52.65:33071 (local address 127.19.52.126:35849)
0812 06:59:15.672654 (+ 257us) server_negotiation.cc:206] Beginning negotiation
0812 06:59:15.672659 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 06:59:15.673769 (+ 1110us) server_negotiation.cc:407] Connection header received
0812 06:59:15.675230 (+ 1461us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:59:15.675234 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:59:15.675300 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:59:15.675412 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:59:15.676742 (+ 1330us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:59:15.677497 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:59:15.679251 (+ 1754us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:59:15.679518 (+ 267us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:59:15.679638 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:59:15.682708 (+ 3070us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:59:15.682716 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:59:15.682722 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:59:15.682768 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:59:15.684480 (+ 1712us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:59:15.685348 (+ 868us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:59:15.685350 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:59:15.685352 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:59:15.685436 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:59:15.685953 (+ 517us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:59:15.685957 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:59:15.685958 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:59:15.686181 (+ 223us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:59:15.686272 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0812 06:59:15.687104 (+ 832us) server_negotiation.cc:299] Negotiation successful
0812 06:59:15.687237 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":85}
I20260812 06:59:15.688697 9431 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:59:15.672820 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.126:35849 (local address 127.19.52.65:33071)
0812 06:59:15.673371 (+ 551us) negotiation.cc:107] Waiting for socket to connect
0812 06:59:15.673433 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0812 06:59:15.674976 (+ 1543us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:59:15.675610 (+ 634us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:59:15.675618 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:59:15.676032 (+ 414us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:59:15.676572 (+ 540us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:59:15.676587 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:59:15.677652 (+ 1065us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:59:15.677676 (+ 24us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:59:15.679117 (+ 1441us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:59:15.679129 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:59:15.679694 (+ 565us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:59:15.679698 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:59:15.679889 (+ 191us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:59:15.680653 (+ 764us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 06:59:15.680678 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:59:15.682476 (+ 1798us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 06:59:15.684706 (+ 2230us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:59:15.684716 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:59:15.684740 (+ 24us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:59:15.685221 (+ 481us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:59:15.685547 (+ 326us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:59:15.685552 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:59:15.685556 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:59:15.685802 (+ 246us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:59:15.686312 (+ 510us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 06:59:15.686322 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 06:59:15.686752 (+ 430us) client_negotiation.cc:769] Sending connection context
0812 06:59:15.686994 (+ 242us) client_negotiation.cc:240] Negotiation successful
0812 06:59:15.687416 (+ 422us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":333,"thread_start_us":128,"threads_started":1}
WARNING: no policy specified for kudu/127.19.52.66@KRBTEST.COM; defaulting to no policy
I20260812 06:59:15.690502 9429 heartbeater.cc:347] Connected to a master server at 127.19.52.126:35849
I20260812 06:59:15.690809 9429 heartbeater.cc:464] Registering TS with master...
I20260812 06:59:15.691388 9429 heartbeater.cc:511] Master 127.19.52.126:35849 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.52.66@KRBTEST.COM" created.
I20260812 06:59:15.693470 9202 ts_manager.cc:194] Registered new tserver with Master: 669c52c47fb144edbe5fb712a33cbcf0 (127.19.52.65:45275)
I20260812 06:59:15.694839 9202 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.52.65@KRBTEST.COM'} at 127.19.52.65:33071
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.52.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786517687384214-19665-0/krb5kdc/kudu_127.19.52.66.keytab.
Entry for principal kudu/127.19.52.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786517687384214-19665-0/krb5kdc/kudu_127.19.52.66.keytab.
I20260812 06:59:15.713151 19665 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu
/tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786517687384214-19665-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786517687384214-19665-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786517687384214-19665-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786517687384214-19665-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.52.66:0
--local_ip_for_outbound_sockets=127.19.52.66
--webserver_interface=127.19.52.66
--webserver_port=0
--tserver_master_addrs=127.19.52.126:35849
--keytab_file=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786517687384214-19665-0/krb5kdc/kudu_127.19.52.66.keytab
--principal=kudu/127.19.52.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.52.84:36075
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786517687384214-19665-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786517687384214-19665-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786517687384214-19665-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 06:59:15.835628 9434 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:59:15.835954 9434 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:59:15.836009 9434 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 06:59:15.840322 9434 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 06:59:15.840413 9434 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 06:59:15.840547 9434 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.52.66
I20260812 06:59:15.846916 9434 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.52.84:36075
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786517687384214-19665-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786517687384214-19665-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786517687384214-19665-0/krb5kdc/kudu_127.19.52.66.keytab
--principal=kudu/127.19.52.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.52.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786517687384214-19665-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.19.52.66
--webserver_port=0
--tserver_master_addrs=127.19.52.126:35849
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9434
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.52.66
--log_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786517687384214-19665-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 06:15:54 UTC on ccycloud.quasar-rnfzce.root.comops.site
build id 4581989
I20260812 06:59:15.848027 9434 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:59:15.849107 9434 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 06:59:15.859345 9441 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:59:15.859328 9440 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:59:15.859565 9434 server_base.cc:1034] running on GCE node
W20260812 06:59:15.859328 9443 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:59:15.860199 9434 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:59:15.861011 9434 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:59:15.862198 9434 hybrid_clock.cc:648] HybridClock initialized: now 1786517955862177 us; error 32 us; skew 500 ppm
Aug 12 06:59:15 dist-test-slave-65mx krb5kdc[9144](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517955, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.52.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 06:59:15.865453 9434 init.cc:377] Logged in from keytab as kudu/127.19.52.66@KRBTEST.COM (short username kudu)
I20260812 06:59:15.866431 9434 webserver.cc:460] Webserver started at http://127.19.52.66:46391/ using document root <none> and password file <none>
I20260812 06:59:15.867063 9434 fs_manager.cc:359] Metadata directory not provided
I20260812 06:59:15.867168 9434 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:59:15.867419 9434 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 06:59:15.869753 9434 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786517687384214-19665-0/minicluster-data/ts-1/data/instance:
uuid: "425eb5090ac24183a7caff4edb2250da"
format_stamp: "Formatted at 2026-08-12 06:59:15 on dist-test-slave-65mx"
server_key: "9d0f80c19daba7ce8cf694ca1f373482"
server_key_iv: "f48109ee89d362eba2b3eab5863566d5"
server_key_version: "encryptionkey@0"
I20260812 06:59:15.870460 9434 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786517687384214-19665-0/minicluster-data/ts-1/wal/instance:
uuid: "425eb5090ac24183a7caff4edb2250da"
format_stamp: "Formatted at 2026-08-12 06:59:15 on dist-test-slave-65mx"
server_key: "9d0f80c19daba7ce8cf694ca1f373482"
server_key_iv: "f48109ee89d362eba2b3eab5863566d5"
server_key_version: "encryptionkey@0"
I20260812 06:59:15.874819 9434 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260812 06:59:15.877843 9454 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 06:59:15.879276 9434 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 06:59:15.879446 9434 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786517687384214-19665-0/minicluster-data/ts-1/data,/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786517687384214-19665-0/minicluster-data/ts-1/wal
uuid: "425eb5090ac24183a7caff4edb2250da"
format_stamp: "Formatted at 2026-08-12 06:59:15 on dist-test-slave-65mx"
server_key: "9d0f80c19daba7ce8cf694ca1f373482"
server_key_iv: "f48109ee89d362eba2b3eab5863566d5"
server_key_version: "encryptionkey@0"
I20260812 06:59:15.879577 9434 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786517687384214-19665-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786517687384214-19665-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786517687384214-19665-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:59:15.895777 9434 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 06:59:15.899245 9434 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 06:59:15.899464 9434 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:59:15.900097 9434 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 06:59:15.901015 9434 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 06:59:15.901086 9434 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 06:59:15.901160 9434 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 06:59:15.901206 9434 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 06:59:15.910100 9434 rpc_server.cc:307] RPC server started. Bound to: 127.19.52.66:34757
I20260812 06:59:15.910125 9587 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.52.66:34757 every 8 connection(s)
I20260812 06:59:15.911033 9434 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786517687384214-19665-0/minicluster-data/ts-1/data/info.pb
I20260812 06:59:15.911940 19665 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu as pid 9434
I20260812 06:59:15.912109 19665 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786517687384214-19665-0/minicluster-data/ts-1/wal/instance
I20260812 06:59:15.912448 19665 external_mini_cluster.cc:1428] Setting key b725aaebb7818de4a6dcbee0351d1ea8
Aug 12 06:59:15 dist-test-slave-65mx krb5kdc[9144](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517955, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.52.66@KRBTEST.COM for kudu/127.19.52.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.19.52.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.52.67@KRBTEST.COM" created.
I20260812 06:59:15.927837 9247 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:59:15.913002 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.52.66:54651 (local address 127.19.52.126:35849)
0812 06:59:15.913216 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0812 06:59:15.913220 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 06:59:15.914144 (+ 924us) server_negotiation.cc:407] Connection header received
0812 06:59:15.916143 (+ 1999us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:59:15.916146 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:59:15.916191 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:59:15.916262 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:59:15.917728 (+ 1466us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:59:15.918344 (+ 616us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:59:15.920284 (+ 1940us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:59:15.920503 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:59:15.920667 (+ 164us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:59:15.923682 (+ 3015us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:59:15.923686 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:59:15.923688 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:59:15.923717 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:59:15.925336 (+ 1619us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:59:15.925910 (+ 574us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:59:15.925913 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:59:15.925915 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:59:15.926005 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:59:15.926500 (+ 495us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:59:15.926503 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:59:15.926504 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:59:15.926738 (+ 234us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:59:15.926860 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0812 06:59:15.927524 (+ 664us) server_negotiation.cc:299] Negotiation successful
0812 06:59:15.927666 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260812 06:59:15.928879 9590 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:59:15.913405 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.126:35849 (local address 127.19.52.66:54651)
0812 06:59:15.913965 (+ 560us) negotiation.cc:107] Waiting for socket to connect
0812 06:59:15.914026 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0812 06:59:15.915895 (+ 1869us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:59:15.916481 (+ 586us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:59:15.916492 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:59:15.917062 (+ 570us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:59:15.917562 (+ 500us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:59:15.917576 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:59:15.918525 (+ 949us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:59:15.918532 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:59:15.920132 (+ 1600us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:59:15.920147 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:59:15.920732 (+ 585us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:59:15.920739 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:59:15.920984 (+ 245us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:59:15.921712 (+ 728us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 06:59:15.921740 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:59:15.923476 (+ 1736us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 06:59:15.925470 (+ 1994us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:59:15.925476 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:59:15.925496 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:59:15.925792 (+ 296us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:59:15.926140 (+ 348us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:59:15.926145 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:59:15.926148 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:59:15.926338 (+ 190us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:59:15.926861 (+ 523us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 06:59:15.926870 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 06:59:15.927225 (+ 355us) client_negotiation.cc:769] Sending connection context
0812 06:59:15.927475 (+ 250us) client_negotiation.cc:240] Negotiation successful
0812 06:59:15.927850 (+ 375us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":306,"thread_start_us":119,"threads_started":1}
I20260812 06:59:15.930336 9588 heartbeater.cc:347] Connected to a master server at 127.19.52.126:35849
I20260812 06:59:15.930598 9588 heartbeater.cc:464] Registering TS with master...
I20260812 06:59:15.931129 9588 heartbeater.cc:511] Master 127.19.52.126:35849 requested a full tablet report, sending...
I20260812 06:59:15.932260 9202 ts_manager.cc:194] Registered new tserver with Master: 425eb5090ac24183a7caff4edb2250da (127.19.52.66:34757)
I20260812 06:59:15.932924 9202 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.52.66@KRBTEST.COM'} at 127.19.52.66:54651
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.52.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786517687384214-19665-0/krb5kdc/kudu_127.19.52.67.keytab.
Entry for principal kudu/127.19.52.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786517687384214-19665-0/krb5kdc/kudu_127.19.52.67.keytab.
I20260812 06:59:15.945879 19665 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu
/tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786517687384214-19665-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786517687384214-19665-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786517687384214-19665-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786517687384214-19665-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.52.67:0
--local_ip_for_outbound_sockets=127.19.52.67
--webserver_interface=127.19.52.67
--webserver_port=0
--tserver_master_addrs=127.19.52.126:35849
--keytab_file=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786517687384214-19665-0/krb5kdc/kudu_127.19.52.67.keytab
--principal=kudu/127.19.52.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.52.84:36075
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786517687384214-19665-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786517687384214-19665-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786517687384214-19665-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 06:59:16.065075 9593 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:59:16.065333 9593 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:59:16.065377 9593 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 06:59:16.069474 9593 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 06:59:16.069538 9593 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 06:59:16.069597 9593 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.52.67
I20260812 06:59:16.074968 9593 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.52.84:36075
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786517687384214-19665-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786517687384214-19665-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786517687384214-19665-0/krb5kdc/kudu_127.19.52.67.keytab
--principal=kudu/127.19.52.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.52.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786517687384214-19665-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.19.52.67
--webserver_port=0
--tserver_master_addrs=127.19.52.126:35849
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9593
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.52.67
--log_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786517687384214-19665-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 06:15:54 UTC on ccycloud.quasar-rnfzce.root.comops.site
build id 4581989
I20260812 06:59:16.076013 9593 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:59:16.077088 9593 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 06:59:16.087216 9603 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:59:16.087216 9598 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:59:16.087257 9599 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:59:16.087507 9593 server_base.cc:1034] running on GCE node
I20260812 06:59:16.088100 9593 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:59:16.088845 9593 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:59:16.090101 9593 hybrid_clock.cc:648] HybridClock initialized: now 1786517956090070 us; error 40 us; skew 500 ppm
Aug 12 06:59:16 dist-test-slave-65mx krb5kdc[9144](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517956, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.52.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 06:59:16.093191 9593 init.cc:377] Logged in from keytab as kudu/127.19.52.67@KRBTEST.COM (short username kudu)
I20260812 06:59:16.094276 9593 webserver.cc:460] Webserver started at http://127.19.52.67:39281/ using document root <none> and password file <none>
I20260812 06:59:16.094928 9593 fs_manager.cc:359] Metadata directory not provided
I20260812 06:59:16.095037 9593 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:59:16.095286 9593 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 06:59:16.097820 9593 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786517687384214-19665-0/minicluster-data/ts-2/data/instance:
uuid: "937242cd30ec477ba629ad0261d99255"
format_stamp: "Formatted at 2026-08-12 06:59:16 on dist-test-slave-65mx"
server_key: "235704062d1b17ce80c1d43eb4679ea9"
server_key_iv: "d7817250db89e0c83d1f60d00e7c7372"
server_key_version: "encryptionkey@0"
I20260812 06:59:16.098480 9593 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786517687384214-19665-0/minicluster-data/ts-2/wal/instance:
uuid: "937242cd30ec477ba629ad0261d99255"
format_stamp: "Formatted at 2026-08-12 06:59:16 on dist-test-slave-65mx"
server_key: "235704062d1b17ce80c1d43eb4679ea9"
server_key_iv: "d7817250db89e0c83d1f60d00e7c7372"
server_key_version: "encryptionkey@0"
I20260812 06:59:16.102844 9593 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260812 06:59:16.105557 9613 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 06:59:16.106915 9593 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 06:59:16.107095 9593 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786517687384214-19665-0/minicluster-data/ts-2/data,/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786517687384214-19665-0/minicluster-data/ts-2/wal
uuid: "937242cd30ec477ba629ad0261d99255"
format_stamp: "Formatted at 2026-08-12 06:59:16 on dist-test-slave-65mx"
server_key: "235704062d1b17ce80c1d43eb4679ea9"
server_key_iv: "d7817250db89e0c83d1f60d00e7c7372"
server_key_version: "encryptionkey@0"
I20260812 06:59:16.107223 9593 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786517687384214-19665-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786517687384214-19665-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786517687384214-19665-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:59:16.128903 9593 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 06:59:16.132135 9593 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 06:59:16.132380 9593 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:59:16.133092 9593 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 06:59:16.134024 9593 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 06:59:16.134099 9593 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 06:59:16.134173 9593 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 06:59:16.134217 9593 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 06:59:16.144019 9593 rpc_server.cc:307] RPC server started. Bound to: 127.19.52.67:42275
I20260812 06:59:16.144059 9742 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.52.67:42275 every 8 connection(s)
I20260812 06:59:16.145035 9593 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786517687384214-19665-0/minicluster-data/ts-2/data/info.pb
I20260812 06:59:16.154273 19665 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu as pid 9593
I20260812 06:59:16.154407 19665 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786517687384214-19665-0/minicluster-data/ts-2/wal/instance
I20260812 06:59:16.154749 19665 external_mini_cluster.cc:1428] Setting key 097d2e2c07313de4aaebfe149e4db483
Aug 12 06:59:16 dist-test-slave-65mx krb5kdc[9144](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517956, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.52.67@KRBTEST.COM for kudu/127.19.52.126@KRBTEST.COM
I20260812 06:59:16.160130 9247 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:59:16.146957 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.52.67:38815 (local address 127.19.52.126:35849)
0812 06:59:16.147121 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0812 06:59:16.147125 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 06:59:16.148411 (+ 1286us) server_negotiation.cc:407] Connection header received
0812 06:59:16.149621 (+ 1210us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:59:16.149624 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:59:16.149699 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:59:16.149803 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:59:16.151081 (+ 1278us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:59:16.151632 (+ 551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:59:16.152871 (+ 1239us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:59:16.153109 (+ 238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:59:16.153229 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:59:16.156165 (+ 2936us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:59:16.156172 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:59:16.156175 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:59:16.156205 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:59:16.157861 (+ 1656us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:59:16.158412 (+ 551us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:59:16.158415 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:59:16.158416 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:59:16.158506 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:59:16.158884 (+ 378us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:59:16.158886 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:59:16.158887 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:59:16.159087 (+ 200us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:59:16.159173 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0812 06:59:16.159877 (+ 704us) server_negotiation.cc:299] Negotiation successful
0812 06:59:16.159996 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260812 06:59:16.161159 9745 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:59:16.147345 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.126:35849 (local address 127.19.52.67:38815)
0812 06:59:16.148242 (+ 897us) negotiation.cc:107] Waiting for socket to connect
0812 06:59:16.148287 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0812 06:59:16.149188 (+ 901us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:59:16.150000 (+ 812us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:59:16.150008 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:59:16.150423 (+ 415us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:59:16.150911 (+ 488us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:59:16.150924 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:59:16.151774 (+ 850us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:59:16.151779 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:59:16.152725 (+ 946us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:59:16.152736 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:59:16.153290 (+ 554us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:59:16.153295 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:59:16.153475 (+ 180us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:59:16.154160 (+ 685us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 06:59:16.154177 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:59:16.156004 (+ 1827us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 06:59:16.157993 (+ 1989us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:59:16.157999 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:59:16.158012 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:59:16.158296 (+ 284us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:59:16.158623 (+ 327us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:59:16.158625 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:59:16.158627 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:59:16.158781 (+ 154us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:59:16.159223 (+ 442us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 06:59:16.159229 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 06:59:16.159611 (+ 382us) client_negotiation.cc:769] Sending connection context
0812 06:59:16.159803 (+ 192us) client_negotiation.cc:240] Negotiation successful
0812 06:59:16.160100 (+ 297us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":708,"thread_start_us":555,"threads_started":1}
I20260812 06:59:16.162477 9743 heartbeater.cc:347] Connected to a master server at 127.19.52.126:35849
I20260812 06:59:16.162756 9743 heartbeater.cc:464] Registering TS with master...
I20260812 06:59:16.163296 9743 heartbeater.cc:511] Master 127.19.52.126:35849 requested a full tablet report, sending...
I20260812 06:59:16.164662 9202 ts_manager.cc:194] Registered new tserver with Master: 937242cd30ec477ba629ad0261d99255 (127.19.52.67:42275)
I20260812 06:59:16.165315 9202 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.52.67@KRBTEST.COM'} at 127.19.52.67:38815
I20260812 06:59:16.169248 19665 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 06:59:16.178999 9247 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:59:16.171618 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54358 (local address 127.19.52.126:35849)
0812 06:59:16.171776 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0812 06:59:16.171780 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 06:59:16.171797 (+ 17us) server_negotiation.cc:407] Connection header received
0812 06:59:16.171837 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:59:16.171839 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:59:16.171880 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:59:16.171973 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:59:16.172757 (+ 784us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:59:16.173319 (+ 562us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:59:16.174152 (+ 833us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:59:16.174367 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:59:16.174462 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:59:16.175885 (+ 1423us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:59:16.175888 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:59:16.175891 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:59:16.175914 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:59:16.177370 (+ 1456us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:59:16.177829 (+ 459us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:59:16.177832 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:59:16.177833 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:59:16.177918 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:59:16.178219 (+ 301us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:59:16.178221 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:59:16.178222 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:59:16.178448 (+ 226us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:59:16.178554 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0812 06:59:16.178769 (+ 215us) server_negotiation.cc:299] Negotiation successful
0812 06:59:16.178874 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260812 06:59:16.182333 9202 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54358:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 06:59:16.185640 9202 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 06:59:16.200629 9758 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:59:16.194447 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.66:34757 (local address 127.0.0.1:50776)
0812 06:59:16.195254 (+ 807us) negotiation.cc:107] Waiting for socket to connect
0812 06:59:16.195268 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 06:59:16.195353 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:59:16.196592 (+ 1239us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:59:16.196597 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:59:16.196615 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 06:59:16.196703 (+ 88us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:59:16.196709 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:59:16.198019 (+ 1310us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:59:16.198023 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:59:16.199148 (+ 1125us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:59:16.199158 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:59:16.199986 (+ 828us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:59:16.199990 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:59:16.200115 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:59:16.200153 (+ 38us) client_negotiation.cc:769] Sending connection context
0812 06:59:16.200294 (+ 141us) client_negotiation.cc:240] Negotiation successful
0812 06:59:16.200418 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":743,"mutex_wait_us":62,"spinlock_wait_cycles":8576,"thread_start_us":127,"threads_started":1}
I20260812 06:59:16.200785 9759 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:59:16.194546 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50932 (local address 127.19.52.67:42275)
0812 06:59:16.195214 (+ 668us) server_negotiation.cc:206] Beginning negotiation
0812 06:59:16.195226 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0812 06:59:16.195250 (+ 24us) server_negotiation.cc:407] Connection header received
0812 06:59:16.195334 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:59:16.195340 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:59:16.195580 (+ 240us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:59:16.195737 (+ 157us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 06:59:16.196260 (+ 523us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:59:16.197540 (+ 1280us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:59:16.199327 (+ 1787us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:59:16.200175 (+ 848us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:59:16.200324 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:59:16.200452 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0812 06:59:16.200508 (+ 56us) server_negotiation.cc:299] Negotiation successful
0812 06:59:16.200635 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":559,"thread_start_us":146,"threads_started":1}
I20260812 06:59:16.201485 9760 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:59:16.195157 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.65:45275 (local address 127.0.0.1:48394)
0812 06:59:16.195972 (+ 815us) negotiation.cc:107] Waiting for socket to connect
0812 06:59:16.195999 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0812 06:59:16.196091 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:59:16.196782 (+ 691us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:59:16.196786 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:59:16.196804 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 06:59:16.196874 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:59:16.196878 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:59:16.198008 (+ 1130us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:59:16.198011 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:59:16.199913 (+ 1902us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:59:16.199926 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:59:16.201106 (+ 1180us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:59:16.201110 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:59:16.201195 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:59:16.201217 (+ 22us) client_negotiation.cc:769] Sending connection context
0812 06:59:16.201283 (+ 66us) client_negotiation.cc:240] Negotiation successful
0812 06:59:16.201354 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":748,"mutex_wait_us":3,"thread_start_us":100,"threads_started":1}
I20260812 06:59:16.201536 9762 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:59:16.195900 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48394 (local address 127.19.52.65:45275)
0812 06:59:16.196207 (+ 307us) server_negotiation.cc:206] Beginning negotiation
0812 06:59:16.196216 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 06:59:16.196240 (+ 24us) server_negotiation.cc:407] Connection header received
0812 06:59:16.196311 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:59:16.196314 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:59:16.196453 (+ 139us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:59:16.196642 (+ 189us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 06:59:16.197031 (+ 389us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:59:16.197874 (+ 843us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:59:16.200064 (+ 2190us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:59:16.200982 (+ 918us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:59:16.201111 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:59:16.201248 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0812 06:59:16.201306 (+ 58us) server_negotiation.cc:299] Negotiation successful
0812 06:59:16.201388 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":192,"thread_start_us":83,"threads_started":1}
I20260812 06:59:16.200678 9756 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:59:16.194409 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.67:42275 (local address 127.0.0.1:50932)
0812 06:59:16.194776 (+ 367us) negotiation.cc:107] Waiting for socket to connect
0812 06:59:16.194823 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0812 06:59:16.194944 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:59:16.195947 (+ 1003us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:59:16.195953 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:59:16.195973 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 06:59:16.196130 (+ 157us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:59:16.196137 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:59:16.197684 (+ 1547us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:59:16.197687 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:59:16.199181 (+ 1494us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:59:16.199192 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:59:16.200301 (+ 1109us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:59:16.200305 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:59:16.200399 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:59:16.200419 (+ 20us) client_negotiation.cc:769] Sending connection context
0812 06:59:16.200480 (+ 61us) client_negotiation.cc:240] Negotiation successful
0812 06:59:16.200569 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":190,"mutex_wait_us":3,"thread_start_us":106,"threads_started":1}
I20260812 06:59:16.201977 9761 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:59:16.195418 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50776 (local address 127.19.52.66:34757)
0812 06:59:16.195746 (+ 328us) server_negotiation.cc:206] Beginning negotiation
0812 06:59:16.195758 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0812 06:59:16.195781 (+ 23us) server_negotiation.cc:407] Connection header received
0812 06:59:16.196218 (+ 437us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:59:16.196225 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:59:16.196409 (+ 184us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:59:16.196598 (+ 189us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 06:59:16.196806 (+ 208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:59:16.197849 (+ 1043us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:59:16.199276 (+ 1427us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:59:16.199849 (+ 573us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:59:16.199974 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:59:16.200148 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0812 06:59:16.201265 (+ 1117us) server_negotiation.cc:299] Negotiation successful
0812 06:59:16.201374 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":227,"thread_start_us":132,"threads_started":1}
I20260812 06:59:16.204260 9350 tablet_service.cc:1467] Processing CreateTablet for tablet bedae456d1454337a55416c65d0e3d4c (DEFAULT_TABLE table=test-table [id=22379c0e9cf54f079ae2f30fb063a5ad]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 06:59:16.204463 9509 tablet_service.cc:1467] Processing CreateTablet for tablet bedae456d1454337a55416c65d0e3d4c (DEFAULT_TABLE table=test-table [id=22379c0e9cf54f079ae2f30fb063a5ad]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 06:59:16.204452 9667 tablet_service.cc:1467] Processing CreateTablet for tablet bedae456d1454337a55416c65d0e3d4c (DEFAULT_TABLE table=test-table [id=22379c0e9cf54f079ae2f30fb063a5ad]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 06:59:16.205418 9350 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bedae456d1454337a55416c65d0e3d4c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 06:59:16.205516 9509 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bedae456d1454337a55416c65d0e3d4c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 06:59:16.205528 9667 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bedae456d1454337a55416c65d0e3d4c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 06:59:16.212225 9763 tablet_bootstrap.cc:492] T bedae456d1454337a55416c65d0e3d4c P 669c52c47fb144edbe5fb712a33cbcf0: Bootstrap starting.
I20260812 06:59:16.213747 9765 tablet_bootstrap.cc:492] T bedae456d1454337a55416c65d0e3d4c P 937242cd30ec477ba629ad0261d99255: Bootstrap starting.
I20260812 06:59:16.213878 9767 tablet_bootstrap.cc:492] T bedae456d1454337a55416c65d0e3d4c P 425eb5090ac24183a7caff4edb2250da: Bootstrap starting.
I20260812 06:59:16.214692 9763 tablet_bootstrap.cc:654] T bedae456d1454337a55416c65d0e3d4c P 669c52c47fb144edbe5fb712a33cbcf0: Neither blocks nor log segments found. Creating new log.
I20260812 06:59:16.215452 9763 log.cc:824] T bedae456d1454337a55416c65d0e3d4c P 669c52c47fb144edbe5fb712a33cbcf0: Log is configured to *not* fsync() on all Append() calls
I20260812 06:59:16.216017 9765 tablet_bootstrap.cc:654] T bedae456d1454337a55416c65d0e3d4c P 937242cd30ec477ba629ad0261d99255: Neither blocks nor log segments found. Creating new log.
I20260812 06:59:16.216630 9767 tablet_bootstrap.cc:654] T bedae456d1454337a55416c65d0e3d4c P 425eb5090ac24183a7caff4edb2250da: Neither blocks nor log segments found. Creating new log.
I20260812 06:59:16.217471 9767 log.cc:824] T bedae456d1454337a55416c65d0e3d4c P 425eb5090ac24183a7caff4edb2250da: Log is configured to *not* fsync() on all Append() calls
I20260812 06:59:16.217470 9765 log.cc:824] T bedae456d1454337a55416c65d0e3d4c P 937242cd30ec477ba629ad0261d99255: Log is configured to *not* fsync() on all Append() calls
I20260812 06:59:16.219095 9763 tablet_bootstrap.cc:492] T bedae456d1454337a55416c65d0e3d4c P 669c52c47fb144edbe5fb712a33cbcf0: No bootstrap required, opened a new log
I20260812 06:59:16.219349 9767 tablet_bootstrap.cc:492] T bedae456d1454337a55416c65d0e3d4c P 425eb5090ac24183a7caff4edb2250da: No bootstrap required, opened a new log
I20260812 06:59:16.219343 9763 ts_tablet_manager.cc:1397] T bedae456d1454337a55416c65d0e3d4c P 669c52c47fb144edbe5fb712a33cbcf0: Time spent bootstrapping tablet: real 0.007s user 0.004s sys 0.000s
I20260812 06:59:16.219551 9767 ts_tablet_manager.cc:1397] T bedae456d1454337a55416c65d0e3d4c P 425eb5090ac24183a7caff4edb2250da: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.004s
I20260812 06:59:16.219699 9765 tablet_bootstrap.cc:492] T bedae456d1454337a55416c65d0e3d4c P 937242cd30ec477ba629ad0261d99255: No bootstrap required, opened a new log
I20260812 06:59:16.219908 9765 ts_tablet_manager.cc:1397] T bedae456d1454337a55416c65d0e3d4c P 937242cd30ec477ba629ad0261d99255: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.004s
I20260812 06:59:16.222630 9767 raft_consensus.cc:348] T bedae456d1454337a55416c65d0e3d4c P 425eb5090ac24183a7caff4edb2250da [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "937242cd30ec477ba629ad0261d99255" member_type: VOTER last_known_addr { host: "127.19.52.67" port: 42275 } } peers { permanent_uuid: "425eb5090ac24183a7caff4edb2250da" member_type: VOTER last_known_addr { host: "127.19.52.66" port: 34757 } } peers { permanent_uuid: "669c52c47fb144edbe5fb712a33cbcf0" member_type: VOTER last_known_addr { host: "127.19.52.65" port: 45275 } }
I20260812 06:59:16.222918 9767 raft_consensus.cc:374] T bedae456d1454337a55416c65d0e3d4c P 425eb5090ac24183a7caff4edb2250da [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 06:59:16.222975 9767 raft_consensus.cc:729] T bedae456d1454337a55416c65d0e3d4c P 425eb5090ac24183a7caff4edb2250da [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 425eb5090ac24183a7caff4edb2250da, State: Initialized, Role: FOLLOWER
I20260812 06:59:16.223449 9767 consensus_queue.cc:260] T bedae456d1454337a55416c65d0e3d4c P 425eb5090ac24183a7caff4edb2250da [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "937242cd30ec477ba629ad0261d99255" member_type: VOTER last_known_addr { host: "127.19.52.67" port: 42275 } } peers { permanent_uuid: "425eb5090ac24183a7caff4edb2250da" member_type: VOTER last_known_addr { host: "127.19.52.66" port: 34757 } } peers { permanent_uuid: "669c52c47fb144edbe5fb712a33cbcf0" member_type: VOTER last_known_addr { host: "127.19.52.65" port: 45275 } }
I20260812 06:59:16.224167 9588 heartbeater.cc:503] Master 127.19.52.126:35849 was elected leader, sending a full tablet report...
I20260812 06:59:16.224330 9765 raft_consensus.cc:348] T bedae456d1454337a55416c65d0e3d4c P 937242cd30ec477ba629ad0261d99255 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "937242cd30ec477ba629ad0261d99255" member_type: VOTER last_known_addr { host: "127.19.52.67" port: 42275 } } peers { permanent_uuid: "425eb5090ac24183a7caff4edb2250da" member_type: VOTER last_known_addr { host: "127.19.52.66" port: 34757 } } peers { permanent_uuid: "669c52c47fb144edbe5fb712a33cbcf0" member_type: VOTER last_known_addr { host: "127.19.52.65" port: 45275 } }
I20260812 06:59:16.224586 9767 ts_tablet_manager.cc:1428] T bedae456d1454337a55416c65d0e3d4c P 425eb5090ac24183a7caff4edb2250da: Time spent starting tablet: real 0.005s user 0.006s sys 0.001s
I20260812 06:59:16.224645 9765 raft_consensus.cc:374] T bedae456d1454337a55416c65d0e3d4c P 937242cd30ec477ba629ad0261d99255 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 06:59:16.224714 9765 raft_consensus.cc:729] T bedae456d1454337a55416c65d0e3d4c P 937242cd30ec477ba629ad0261d99255 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 937242cd30ec477ba629ad0261d99255, State: Initialized, Role: FOLLOWER
I20260812 06:59:16.225236 9765 consensus_queue.cc:260] T bedae456d1454337a55416c65d0e3d4c P 937242cd30ec477ba629ad0261d99255 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "937242cd30ec477ba629ad0261d99255" member_type: VOTER last_known_addr { host: "127.19.52.67" port: 42275 } } peers { permanent_uuid: "425eb5090ac24183a7caff4edb2250da" member_type: VOTER last_known_addr { host: "127.19.52.66" port: 34757 } } peers { permanent_uuid: "669c52c47fb144edbe5fb712a33cbcf0" member_type: VOTER last_known_addr { host: "127.19.52.65" port: 45275 } }
I20260812 06:59:16.225966 9743 heartbeater.cc:503] Master 127.19.52.126:35849 was elected leader, sending a full tablet report...
I20260812 06:59:16.226352 9765 ts_tablet_manager.cc:1428] T bedae456d1454337a55416c65d0e3d4c P 937242cd30ec477ba629ad0261d99255: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260812 06:59:16.225757 9763 raft_consensus.cc:348] T bedae456d1454337a55416c65d0e3d4c P 669c52c47fb144edbe5fb712a33cbcf0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "937242cd30ec477ba629ad0261d99255" member_type: VOTER last_known_addr { host: "127.19.52.67" port: 42275 } } peers { permanent_uuid: "425eb5090ac24183a7caff4edb2250da" member_type: VOTER last_known_addr { host: "127.19.52.66" port: 34757 } } peers { permanent_uuid: "669c52c47fb144edbe5fb712a33cbcf0" member_type: VOTER last_known_addr { host: "127.19.52.65" port: 45275 } }
I20260812 06:59:16.227166 9763 raft_consensus.cc:374] T bedae456d1454337a55416c65d0e3d4c P 669c52c47fb144edbe5fb712a33cbcf0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 06:59:16.227241 9763 raft_consensus.cc:729] T bedae456d1454337a55416c65d0e3d4c P 669c52c47fb144edbe5fb712a33cbcf0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 669c52c47fb144edbe5fb712a33cbcf0, State: Initialized, Role: FOLLOWER
I20260812 06:59:16.227722 9763 consensus_queue.cc:260] T bedae456d1454337a55416c65d0e3d4c P 669c52c47fb144edbe5fb712a33cbcf0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "937242cd30ec477ba629ad0261d99255" member_type: VOTER last_known_addr { host: "127.19.52.67" port: 42275 } } peers { permanent_uuid: "425eb5090ac24183a7caff4edb2250da" member_type: VOTER last_known_addr { host: "127.19.52.66" port: 34757 } } peers { permanent_uuid: "669c52c47fb144edbe5fb712a33cbcf0" member_type: VOTER last_known_addr { host: "127.19.52.65" port: 45275 } }
I20260812 06:59:16.228912 9763 ts_tablet_manager.cc:1428] T bedae456d1454337a55416c65d0e3d4c P 669c52c47fb144edbe5fb712a33cbcf0: Time spent starting tablet: real 0.009s user 0.009s sys 0.000s
I20260812 06:59:16.229327 9429 heartbeater.cc:503] Master 127.19.52.126:35849 was elected leader, sending a full tablet report...
W20260812 06:59:16.396093 9744 tablet.cc:2369] T bedae456d1454337a55416c65d0e3d4c P 937242cd30ec477ba629ad0261d99255: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 06:59:16.412438 9589 tablet.cc:2369] T bedae456d1454337a55416c65d0e3d4c P 425eb5090ac24183a7caff4edb2250da: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 06:59:16.421766 9430 tablet.cc:2369] T bedae456d1454337a55416c65d0e3d4c P 669c52c47fb144edbe5fb712a33cbcf0: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 06:59:16.472366 9771 raft_consensus.cc:482] T bedae456d1454337a55416c65d0e3d4c P 425eb5090ac24183a7caff4edb2250da [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 06:59:16.472684 9771 raft_consensus.cc:504] T bedae456d1454337a55416c65d0e3d4c P 425eb5090ac24183a7caff4edb2250da [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "937242cd30ec477ba629ad0261d99255" member_type: VOTER last_known_addr { host: "127.19.52.67" port: 42275 } } peers { permanent_uuid: "425eb5090ac24183a7caff4edb2250da" member_type: VOTER last_known_addr { host: "127.19.52.66" port: 34757 } } peers { permanent_uuid: "669c52c47fb144edbe5fb712a33cbcf0" member_type: VOTER last_known_addr { host: "127.19.52.65" port: 45275 } }
I20260812 06:59:16.473835 9771 leader_election.cc:288] T bedae456d1454337a55416c65d0e3d4c P 425eb5090ac24183a7caff4edb2250da [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 937242cd30ec477ba629ad0261d99255 (127.19.52.67:42275), 669c52c47fb144edbe5fb712a33cbcf0 (127.19.52.65:45275)
I20260812 06:59:16.479715 9762 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:59:16.474826 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.52.66:59887 (local address 127.19.52.65:45275)
0812 06:59:16.474972 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0812 06:59:16.474975 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 06:59:16.475003 (+ 28us) server_negotiation.cc:407] Connection header received
0812 06:59:16.475152 (+ 149us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:59:16.475154 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:59:16.475194 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:59:16.475253 (+ 59us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 06:59:16.475587 (+ 334us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:59:16.476306 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:59:16.478073 (+ 1767us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:59:16.478815 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:59:16.478952 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:59:16.479011 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0812 06:59:16.479480 (+ 469us) server_negotiation.cc:299] Negotiation successful
0812 06:59:16.479570 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260812 06:59:16.479714 9759 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:59:16.474255 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.52.66:39147 (local address 127.19.52.67:42275)
0812 06:59:16.474398 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0812 06:59:16.474402 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 06:59:16.474629 (+ 227us) server_negotiation.cc:407] Connection header received
0812 06:59:16.474714 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:59:16.474719 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:59:16.474786 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:59:16.474914 (+ 128us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 06:59:16.475387 (+ 473us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:59:16.476155 (+ 768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:59:16.478073 (+ 1918us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:59:16.478799 (+ 726us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:59:16.478958 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:59:16.479011 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0812 06:59:16.479476 (+ 465us) server_negotiation.cc:299] Negotiation successful
0812 06:59:16.479567 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260812 06:59:16.479773 9775 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:59:16.474627 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.65:45275 (local address 127.19.52.66:59887)
0812 06:59:16.474914 (+ 287us) negotiation.cc:107] Waiting for socket to connect
0812 06:59:16.474941 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0812 06:59:16.475041 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:59:16.475301 (+ 260us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:59:16.475304 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:59:16.475331 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 06:59:16.475466 (+ 135us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:59:16.475471 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:59:16.476410 (+ 939us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:59:16.476413 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:59:16.477897 (+ 1484us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:59:16.477910 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:59:16.478949 (+ 1039us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:59:16.478953 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:59:16.479290 (+ 337us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:59:16.479336 (+ 46us) client_negotiation.cc:769] Sending connection context
0812 06:59:16.479419 (+ 83us) client_negotiation.cc:240] Negotiation successful
0812 06:59:16.479578 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":202,"mutex_wait_us":204,"thread_start_us":88,"threads_started":1}
I20260812 06:59:16.479781 9774 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:59:16.474175 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.67:42275 (local address 127.19.52.66:39147)
0812 06:59:16.474469 (+ 294us) negotiation.cc:107] Waiting for socket to connect
0812 06:59:16.474485 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0812 06:59:16.474613 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:59:16.475050 (+ 437us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:59:16.475053 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:59:16.475072 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 06:59:16.475221 (+ 149us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:59:16.475227 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:59:16.476268 (+ 1041us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:59:16.476271 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:59:16.477896 (+ 1625us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:59:16.477910 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:59:16.478949 (+ 1039us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:59:16.478953 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:59:16.479283 (+ 330us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:59:16.479330 (+ 47us) client_negotiation.cc:769] Sending connection context
0812 06:59:16.479419 (+ 89us) client_negotiation.cc:240] Negotiation successful
0812 06:59:16.479578 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":217,"mutex_wait_us":230,"thread_start_us":120,"threads_started":1}
I20260812 06:59:16.480748 9374 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bedae456d1454337a55416c65d0e3d4c" candidate_uuid: "425eb5090ac24183a7caff4edb2250da" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "669c52c47fb144edbe5fb712a33cbcf0" is_pre_election: true
I20260812 06:59:16.480973 9691 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bedae456d1454337a55416c65d0e3d4c" candidate_uuid: "425eb5090ac24183a7caff4edb2250da" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "937242cd30ec477ba629ad0261d99255" is_pre_election: true
I20260812 06:59:16.481150 9374 raft_consensus.cc:2454] T bedae456d1454337a55416c65d0e3d4c P 669c52c47fb144edbe5fb712a33cbcf0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 425eb5090ac24183a7caff4edb2250da in term 0.
I20260812 06:59:16.481248 9691 raft_consensus.cc:2454] T bedae456d1454337a55416c65d0e3d4c P 937242cd30ec477ba629ad0261d99255 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 425eb5090ac24183a7caff4edb2250da in term 0.
I20260812 06:59:16.481726 9457 leader_election.cc:302] T bedae456d1454337a55416c65d0e3d4c P 425eb5090ac24183a7caff4edb2250da [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 425eb5090ac24183a7caff4edb2250da, 937242cd30ec477ba629ad0261d99255; no voters:
I20260812 06:59:16.482158 9771 raft_consensus.cc:2781] T bedae456d1454337a55416c65d0e3d4c P 425eb5090ac24183a7caff4edb2250da [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 06:59:16.482299 9771 raft_consensus.cc:482] T bedae456d1454337a55416c65d0e3d4c P 425eb5090ac24183a7caff4edb2250da [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 06:59:16.482353 9771 raft_consensus.cc:3037] T bedae456d1454337a55416c65d0e3d4c P 425eb5090ac24183a7caff4edb2250da [term 0 FOLLOWER]: Advancing to term 1
I20260812 06:59:16.483654 9771 raft_consensus.cc:504] T bedae456d1454337a55416c65d0e3d4c P 425eb5090ac24183a7caff4edb2250da [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "937242cd30ec477ba629ad0261d99255" member_type: VOTER last_known_addr { host: "127.19.52.67" port: 42275 } } peers { permanent_uuid: "425eb5090ac24183a7caff4edb2250da" member_type: VOTER last_known_addr { host: "127.19.52.66" port: 34757 } } peers { permanent_uuid: "669c52c47fb144edbe5fb712a33cbcf0" member_type: VOTER last_known_addr { host: "127.19.52.65" port: 45275 } }
I20260812 06:59:16.484089 9771 leader_election.cc:288] T bedae456d1454337a55416c65d0e3d4c P 425eb5090ac24183a7caff4edb2250da [CANDIDATE]: Term 1 election: Requested vote from peers 937242cd30ec477ba629ad0261d99255 (127.19.52.67:42275), 669c52c47fb144edbe5fb712a33cbcf0 (127.19.52.65:45275)
I20260812 06:59:16.484601 9691 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bedae456d1454337a55416c65d0e3d4c" candidate_uuid: "425eb5090ac24183a7caff4edb2250da" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "937242cd30ec477ba629ad0261d99255"
I20260812 06:59:16.484676 9374 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bedae456d1454337a55416c65d0e3d4c" candidate_uuid: "425eb5090ac24183a7caff4edb2250da" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "669c52c47fb144edbe5fb712a33cbcf0"
I20260812 06:59:16.484730 9691 raft_consensus.cc:3037] T bedae456d1454337a55416c65d0e3d4c P 937242cd30ec477ba629ad0261d99255 [term 0 FOLLOWER]: Advancing to term 1
I20260812 06:59:16.484862 9374 raft_consensus.cc:3037] T bedae456d1454337a55416c65d0e3d4c P 669c52c47fb144edbe5fb712a33cbcf0 [term 0 FOLLOWER]: Advancing to term 1
I20260812 06:59:16.485947 9691 raft_consensus.cc:2454] T bedae456d1454337a55416c65d0e3d4c P 937242cd30ec477ba629ad0261d99255 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 425eb5090ac24183a7caff4edb2250da in term 1.
I20260812 06:59:16.485968 9374 raft_consensus.cc:2454] T bedae456d1454337a55416c65d0e3d4c P 669c52c47fb144edbe5fb712a33cbcf0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 425eb5090ac24183a7caff4edb2250da in term 1.
I20260812 06:59:16.486304 9457 leader_election.cc:302] T bedae456d1454337a55416c65d0e3d4c P 425eb5090ac24183a7caff4edb2250da [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 425eb5090ac24183a7caff4edb2250da, 937242cd30ec477ba629ad0261d99255; no voters:
I20260812 06:59:16.486513 9771 raft_consensus.cc:2781] T bedae456d1454337a55416c65d0e3d4c P 425eb5090ac24183a7caff4edb2250da [term 1 FOLLOWER]: Leader election won for term 1
I20260812 06:59:16.486773 9771 raft_consensus.cc:686] T bedae456d1454337a55416c65d0e3d4c P 425eb5090ac24183a7caff4edb2250da [term 1 LEADER]: Becoming Leader. State: Replica: 425eb5090ac24183a7caff4edb2250da, State: Running, Role: LEADER
I20260812 06:59:16.487178 9771 consensus_queue.cc:237] T bedae456d1454337a55416c65d0e3d4c P 425eb5090ac24183a7caff4edb2250da [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "937242cd30ec477ba629ad0261d99255" member_type: VOTER last_known_addr { host: "127.19.52.67" port: 42275 } } peers { permanent_uuid: "425eb5090ac24183a7caff4edb2250da" member_type: VOTER last_known_addr { host: "127.19.52.66" port: 34757 } } peers { permanent_uuid: "669c52c47fb144edbe5fb712a33cbcf0" member_type: VOTER last_known_addr { host: "127.19.52.65" port: 45275 } }
I20260812 06:59:16.490585 9202 catalog_manager.cc:5495] T bedae456d1454337a55416c65d0e3d4c P 425eb5090ac24183a7caff4edb2250da reported cstate change: term changed from 0 to 1, leader changed from <none> to 425eb5090ac24183a7caff4edb2250da (127.19.52.66). New cstate: current_term: 1 leader_uuid: "425eb5090ac24183a7caff4edb2250da" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "937242cd30ec477ba629ad0261d99255" member_type: VOTER last_known_addr { host: "127.19.52.67" port: 42275 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "425eb5090ac24183a7caff4edb2250da" member_type: VOTER last_known_addr { host: "127.19.52.66" port: 34757 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "669c52c47fb144edbe5fb712a33cbcf0" member_type: VOTER last_known_addr { host: "127.19.52.65" port: 45275 } health_report { overall_health: UNKNOWN } } }
I20260812 06:59:16.567255 9761 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:59:16.563482 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50792 (local address 127.19.52.66:34757)
0812 06:59:16.563638 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0812 06:59:16.563643 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 06:59:16.563659 (+ 16us) server_negotiation.cc:407] Connection header received
0812 06:59:16.563798 (+ 139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:59:16.563800 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:59:16.563862 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:59:16.563998 (+ 136us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 06:59:16.564248 (+ 250us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:59:16.564780 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:59:16.565693 (+ 913us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:59:16.565919 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:59:16.566043 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:59:16.566376 (+ 333us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 06:59:16.566825 (+ 449us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 06:59:16.566979 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0812 06:59:16.567024 (+ 45us) server_negotiation.cc:299] Negotiation successful
0812 06:59:16.567092 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260812 06:59:16.573745 9374 raft_consensus.cc:1261] T bedae456d1454337a55416c65d0e3d4c P 669c52c47fb144edbe5fb712a33cbcf0 [term 1 FOLLOWER]: Refusing update from remote peer 425eb5090ac24183a7caff4edb2250da: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 06:59:16.573745 9691 raft_consensus.cc:1261] T bedae456d1454337a55416c65d0e3d4c P 937242cd30ec477ba629ad0261d99255 [term 1 FOLLOWER]: Refusing update from remote peer 425eb5090ac24183a7caff4edb2250da: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 06:59:16.574536 9771 consensus_queue.cc:1035] T bedae456d1454337a55416c65d0e3d4c P 425eb5090ac24183a7caff4edb2250da [LEADER]: Connected to new peer: Peer: permanent_uuid: "669c52c47fb144edbe5fb712a33cbcf0" member_type: VOTER last_known_addr { host: "127.19.52.65" port: 45275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 06:59:16.574785 9776 consensus_queue.cc:1035] T bedae456d1454337a55416c65d0e3d4c P 425eb5090ac24183a7caff4edb2250da [LEADER]: Connected to new peer: Peer: permanent_uuid: "937242cd30ec477ba629ad0261d99255" member_type: VOTER last_known_addr { host: "127.19.52.67" port: 42275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 06:59:16.583071 9781 mvcc.cc:205] Tried to move back new op lower bound from 7317577550118109184 to 7317577549774229504. Current Snapshot: MvccSnapshot[applied={T|T < 7317577550118109184}]
I20260812 06:59:16.584599 9782 mvcc.cc:205] Tried to move back new op lower bound from 7317577550118109184 to 7317577549774229504. Current Snapshot: MvccSnapshot[applied={T|T < 7317577550118109184}]
I20260812 06:59:16.591194 9202 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54358:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 06:59:16.591357 9202 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54358:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 06:59:16.594077 9202 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 101f82f261d942bc89027bc200f1fab2: Sending DeleteTablet for 3 replicas of tablet bedae456d1454337a55416c65d0e3d4c
I20260812 06:59:16.595053 9509 tablet_service.cc:1514] Processing DeleteTablet for tablet bedae456d1454337a55416c65d0e3d4c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 06:59:16 UTC) from {username='kudu', principal='kudu/127.19.52.126@KRBTEST.COM'} at 127.0.0.1:50776
I20260812 06:59:16.595206 9350 tablet_service.cc:1514] Processing DeleteTablet for tablet bedae456d1454337a55416c65d0e3d4c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 06:59:16 UTC) from {username='kudu', principal='kudu/127.19.52.126@KRBTEST.COM'} at 127.0.0.1:48394
I20260812 06:59:16.595204 9667 tablet_service.cc:1514] Processing DeleteTablet for tablet bedae456d1454337a55416c65d0e3d4c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 06:59:16 UTC) from {username='kudu', principal='kudu/127.19.52.126@KRBTEST.COM'} at 127.0.0.1:50932
I20260812 06:59:16.595845 9791 tablet_replica.cc:330] T bedae456d1454337a55416c65d0e3d4c P 425eb5090ac24183a7caff4edb2250da: stopping tablet replica
I20260812 06:59:16.595849 9793 tablet_replica.cc:330] T bedae456d1454337a55416c65d0e3d4c P 669c52c47fb144edbe5fb712a33cbcf0: stopping tablet replica
I20260812 06:59:16.596208 9791 raft_consensus.cc:2229] T bedae456d1454337a55416c65d0e3d4c P 425eb5090ac24183a7caff4edb2250da [term 1 LEADER]: Raft consensus shutting down.
I20260812 06:59:16.596400 9793 raft_consensus.cc:2229] T bedae456d1454337a55416c65d0e3d4c P 669c52c47fb144edbe5fb712a33cbcf0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 06:59:16.596419 19665 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu with pid 9277
I20260812 06:59:16.596642 9791 raft_consensus.cc:2258] T bedae456d1454337a55416c65d0e3d4c P 425eb5090ac24183a7caff4edb2250da [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 06:59:16.598351 9791 ts_tablet_manager.cc:1905] T bedae456d1454337a55416c65d0e3d4c P 425eb5090ac24183a7caff4edb2250da: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 06:59:16.600903 9795 tablet_replica.cc:330] T bedae456d1454337a55416c65d0e3d4c P 937242cd30ec477ba629ad0261d99255: stopping tablet replica
I20260812 06:59:16.601372 9795 raft_consensus.cc:2229] T bedae456d1454337a55416c65d0e3d4c P 937242cd30ec477ba629ad0261d99255 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 06:59:16.601523 9791 ts_tablet_manager.cc:1918] T bedae456d1454337a55416c65d0e3d4c P 425eb5090ac24183a7caff4edb2250da: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 06:59:16.601632 9791 log.cc:1196] T bedae456d1454337a55416c65d0e3d4c P 425eb5090ac24183a7caff4edb2250da: Deleting WAL directory at /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786517687384214-19665-0/minicluster-data/ts-1/wal/wals/bedae456d1454337a55416c65d0e3d4c
I20260812 06:59:16.601931 9795 raft_consensus.cc:2258] T bedae456d1454337a55416c65d0e3d4c P 937242cd30ec477ba629ad0261d99255 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 06:59:16.601966 9791 ts_tablet_manager.cc:1939] T bedae456d1454337a55416c65d0e3d4c P 425eb5090ac24183a7caff4edb2250da: Deleting consensus metadata
I20260812 06:59:16.603251 9186 catalog_manager.cc:4841] TS 425eb5090ac24183a7caff4edb2250da (127.19.52.66:34757): tablet bedae456d1454337a55416c65d0e3d4c (table test-table [id=22379c0e9cf54f079ae2f30fb063a5ad]) successfully deleted
I20260812 06:59:16.603605 9795 ts_tablet_manager.cc:1905] T bedae456d1454337a55416c65d0e3d4c P 937242cd30ec477ba629ad0261d99255: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 06:59:16.605906 9795 ts_tablet_manager.cc:1918] T bedae456d1454337a55416c65d0e3d4c P 937242cd30ec477ba629ad0261d99255: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 06:59:16.606040 9795 log.cc:1196] T bedae456d1454337a55416c65d0e3d4c P 937242cd30ec477ba629ad0261d99255: Deleting WAL directory at /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786517687384214-19665-0/minicluster-data/ts-2/wal/wals/bedae456d1454337a55416c65d0e3d4c
I20260812 06:59:16.606326 9795 ts_tablet_manager.cc:1939] T bedae456d1454337a55416c65d0e3d4c P 937242cd30ec477ba629ad0261d99255: Deleting consensus metadata
I20260812 06:59:16.607586 9185 catalog_manager.cc:4841] TS 937242cd30ec477ba629ad0261d99255 (127.19.52.67:42275): tablet bedae456d1454337a55416c65d0e3d4c (table test-table [id=22379c0e9cf54f079ae2f30fb063a5ad]) successfully deleted
I20260812 06:59:16.610692 19665 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu with pid 9434
W20260812 06:59:16.610663 9185 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.19.52.65:45275 (error 108)
I20260812 06:59:16.612440 9756 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:59:16.611970 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.65:45275 (local address 127.0.0.1:48400)
0812 06:59:16.612251 (+ 281us) negotiation.cc:107] Waiting for socket to connect
0812 06:59:16.612335 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.52.65:45275: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":170}
W20260812 06:59:16.612703 9185 catalog_manager.cc:4584] TS 669c52c47fb144edbe5fb712a33cbcf0 (127.19.52.65:45275): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet bedae456d1454337a55416c65d0e3d4c: Network error: Client connection negotiation failed: client connection to 127.19.52.65:45275: connect: Connection refused (error 111)
I20260812 06:59:16.619995 19665 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu with pid 9593
I20260812 06:59:16.629878 19665 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu with pid 9158
2026-08-12T06:59:16Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/4 (2778 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/5
2026-08-12T06:59:16Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T06:59:16Z Disabled control of system clock
I20260812 06:59:16.669390 19665 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu
/tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786517687384214-19665-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786517687384214-19665-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786517687384214-19665-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786517687384214-19665-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.52.126:45755
--webserver_interface=127.19.52.126
--webserver_port=0
--builtin_ntp_servers=127.19.52.84:37205
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.19.52.126:45755
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260812 06:59:16.789825 9800 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:59:16.790145 9800 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:59:16.790184 9800 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 06:59:16.794281 9800 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 06:59:16.794356 9800 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 06:59:16.794373 9800 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 06:59:16.794389 9800 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 06:59:16.794404 9800 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 06:59:16.799592 9800 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.52.84:37205
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786517687384214-19665-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786517687384214-19665-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.19.52.126:45755
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.52.126:45755
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786517687384214-19665-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.19.52.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9800
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786517687384214-19665-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 06:15:54 UTC on ccycloud.quasar-rnfzce.root.comops.site
build id 4581989
I20260812 06:59:16.800776 9800 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:59:16.802363 9800 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 06:59:16.811982 9806 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:59:16.811926 9810 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:59:16.811920 9807 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:59:16.812074 9800 server_base.cc:1034] running on GCE node
I20260812 06:59:16.813017 9800 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:59:16.814213 9800 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:59:16.815414 9800 hybrid_clock.cc:648] HybridClock initialized: now 1786517956815380 us; error 53 us; skew 500 ppm
I20260812 06:59:16.817796 9800 webserver.cc:460] Webserver started at http://127.19.52.126:46297/ using document root <none> and password file <none>
I20260812 06:59:16.818441 9800 fs_manager.cc:359] Metadata directory not provided
I20260812 06:59:16.818517 9800 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:59:16.818760 9800 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 06:59:16.821039 9800 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786517687384214-19665-0/minicluster-data/master-0/data/instance:
uuid: "c3fd97b36e534f3496d74e809a8a6056"
format_stamp: "Formatted at 2026-08-12 06:59:16 on dist-test-slave-65mx"
server_key: "1ebf3fe63880801d8d10e97b33f0204c"
server_key_iv: "77b8394adc12107c9f609934483b405e"
server_key_version: "encryptionkey@0"
I20260812 06:59:16.821648 9800 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786517687384214-19665-0/minicluster-data/master-0/wal/instance:
uuid: "c3fd97b36e534f3496d74e809a8a6056"
format_stamp: "Formatted at 2026-08-12 06:59:16 on dist-test-slave-65mx"
server_key: "1ebf3fe63880801d8d10e97b33f0204c"
server_key_iv: "77b8394adc12107c9f609934483b405e"
server_key_version: "encryptionkey@0"
I20260812 06:59:16.825734 9800 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260812 06:59:16.828689 9817 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 06:59:16.830214 9800 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260812 06:59:16.830380 9800 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786517687384214-19665-0/minicluster-data/master-0/data,/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786517687384214-19665-0/minicluster-data/master-0/wal
uuid: "c3fd97b36e534f3496d74e809a8a6056"
format_stamp: "Formatted at 2026-08-12 06:59:16 on dist-test-slave-65mx"
server_key: "1ebf3fe63880801d8d10e97b33f0204c"
server_key_iv: "77b8394adc12107c9f609934483b405e"
server_key_version: "encryptionkey@0"
I20260812 06:59:16.830518 9800 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786517687384214-19665-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786517687384214-19665-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786517687384214-19665-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:59:16.842429 9800 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 06:59:16.843225 9800 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 06:59:16.843438 9800 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:59:16.850977 9800 rpc_server.cc:307] RPC server started. Bound to: 127.19.52.126:45755
I20260812 06:59:16.851006 9874 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.52.126:45755 every 8 connection(s)
I20260812 06:59:16.851996 9800 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786517687384214-19665-0/minicluster-data/master-0/data/info.pb
I20260812 06:59:16.855330 9875 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 06:59:16.856854 19665 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu as pid 9800
I20260812 06:59:16.857028 19665 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786517687384214-19665-0/minicluster-data/master-0/wal/instance
I20260812 06:59:16.857394 19665 external_mini_cluster.cc:1428] Setting key 349515cc12aaaa37a73ac35119da0a66
I20260812 06:59:16.862262 9875 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c3fd97b36e534f3496d74e809a8a6056: Bootstrap starting.
I20260812 06:59:16.865131 9875 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c3fd97b36e534f3496d74e809a8a6056: Neither blocks nor log segments found. Creating new log.
I20260812 06:59:16.865949 9875 log.cc:824] T 00000000000000000000000000000000 P c3fd97b36e534f3496d74e809a8a6056: Log is configured to *not* fsync() on all Append() calls
I20260812 06:59:16.867291 9879 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:59:16.858990 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34676 (local address 127.19.52.126:45755)
0812 06:59:16.859696 (+ 706us) server_negotiation.cc:206] Beginning negotiation
0812 06:59:16.859720 (+ 24us) server_negotiation.cc:399] Waiting for connection header
0812 06:59:16.859757 (+ 37us) server_negotiation.cc:407] Connection header received
0812 06:59:16.860455 (+ 698us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:59:16.860473 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:59:16.860900 (+ 427us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:59:16.861184 (+ 284us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:59:16.861581 (+ 397us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:59:16.862652 (+ 1071us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:59:16.863878 (+ 1226us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:59:16.864349 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:59:16.864490 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:59:16.865406 (+ 916us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:59:16.865411 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:59:16.865424 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 06:59:16.865855 (+ 431us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:59:16.865910 (+ 55us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 06:59:16.865930 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:59:16.866011 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0812 06:59:16.866210 (+ 199us) server_negotiation.cc:299] Negotiation successful
0812 06:59:16.866526 (+ 316us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":293,"thread_start_us":148,"threads_started":1}
I20260812 06:59:16.868053 9875 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c3fd97b36e534f3496d74e809a8a6056: No bootstrap required, opened a new log
I20260812 06:59:16.872119 9875 raft_consensus.cc:348] T 00000000000000000000000000000000 P c3fd97b36e534f3496d74e809a8a6056 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c3fd97b36e534f3496d74e809a8a6056" member_type: VOTER last_known_addr { host: "127.19.52.126" port: 45755 } }
I20260812 06:59:16.872447 9875 raft_consensus.cc:374] T 00000000000000000000000000000000 P c3fd97b36e534f3496d74e809a8a6056 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 06:59:16.872516 9875 raft_consensus.cc:729] T 00000000000000000000000000000000 P c3fd97b36e534f3496d74e809a8a6056 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c3fd97b36e534f3496d74e809a8a6056, State: Initialized, Role: FOLLOWER
I20260812 06:59:16.873000 9875 consensus_queue.cc:260] T 00000000000000000000000000000000 P c3fd97b36e534f3496d74e809a8a6056 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c3fd97b36e534f3496d74e809a8a6056" member_type: VOTER last_known_addr { host: "127.19.52.126" port: 45755 } }
I20260812 06:59:16.873159 9875 raft_consensus.cc:388] T 00000000000000000000000000000000 P c3fd97b36e534f3496d74e809a8a6056 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 06:59:16.873245 9875 raft_consensus.cc:482] T 00000000000000000000000000000000 P c3fd97b36e534f3496d74e809a8a6056 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 06:59:16.873360 9875 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c3fd97b36e534f3496d74e809a8a6056 [term 0 FOLLOWER]: Advancing to term 1
I20260812 06:59:16.874344 9875 raft_consensus.cc:504] T 00000000000000000000000000000000 P c3fd97b36e534f3496d74e809a8a6056 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c3fd97b36e534f3496d74e809a8a6056" member_type: VOTER last_known_addr { host: "127.19.52.126" port: 45755 } }
I20260812 06:59:16.874683 9875 leader_election.cc:302] T 00000000000000000000000000000000 P c3fd97b36e534f3496d74e809a8a6056 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c3fd97b36e534f3496d74e809a8a6056; no voters:
I20260812 06:59:16.875020 9875 leader_election.cc:288] T 00000000000000000000000000000000 P c3fd97b36e534f3496d74e809a8a6056 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 06:59:16.875180 9881 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c3fd97b36e534f3496d74e809a8a6056 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 06:59:16.875432 9881 raft_consensus.cc:686] T 00000000000000000000000000000000 P c3fd97b36e534f3496d74e809a8a6056 [term 1 LEADER]: Becoming Leader. State: Replica: c3fd97b36e534f3496d74e809a8a6056, State: Running, Role: LEADER
I20260812 06:59:16.875757 9881 consensus_queue.cc:237] T 00000000000000000000000000000000 P c3fd97b36e534f3496d74e809a8a6056 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c3fd97b36e534f3496d74e809a8a6056" member_type: VOTER last_known_addr { host: "127.19.52.126" port: 45755 } }
I20260812 06:59:16.876298 9875 sys_catalog.cc:564] T 00000000000000000000000000000000 P c3fd97b36e534f3496d74e809a8a6056 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 06:59:16.877468 9882 sys_catalog.cc:455] T 00000000000000000000000000000000 P c3fd97b36e534f3496d74e809a8a6056 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c3fd97b36e534f3496d74e809a8a6056" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c3fd97b36e534f3496d74e809a8a6056" member_type: VOTER last_known_addr { host: "127.19.52.126" port: 45755 } } }
I20260812 06:59:16.877609 9882 sys_catalog.cc:458] T 00000000000000000000000000000000 P c3fd97b36e534f3496d74e809a8a6056 [sys.catalog]: This master's current role is: LEADER
I20260812 06:59:16.877887 9886 sys_catalog.cc:455] T 00000000000000000000000000000000 P c3fd97b36e534f3496d74e809a8a6056 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c3fd97b36e534f3496d74e809a8a6056. Latest consensus state: current_term: 1 leader_uuid: "c3fd97b36e534f3496d74e809a8a6056" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c3fd97b36e534f3496d74e809a8a6056" member_type: VOTER last_known_addr { host: "127.19.52.126" port: 45755 } } }
I20260812 06:59:16.877998 9886 sys_catalog.cc:458] T 00000000000000000000000000000000 P c3fd97b36e534f3496d74e809a8a6056 [sys.catalog]: This master's current role is: LEADER
I20260812 06:59:16.878114 9894 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 06:59:16.881390 9894 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 06:59:16.886888 9894 catalog_manager.cc:1347] Generated new cluster ID: 289d471d2b7f44f6a7d2f87993ec42ea
I20260812 06:59:16.886976 9894 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 06:59:16.904121 9894 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 06:59:16.905237 9894 catalog_manager.cc:1504] Loading token signing keys...
I20260812 06:59:16.912843 9894 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c3fd97b36e534f3496d74e809a8a6056: Generated new TSK 0
I20260812 06:59:16.913594 9894 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 06:59:16.924636 19665 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu
/tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786517687384214-19665-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786517687384214-19665-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786517687384214-19665-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786517687384214-19665-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.52.65:0
--local_ip_for_outbound_sockets=127.19.52.65
--webserver_interface=127.19.52.65
--webserver_port=0
--tserver_master_addrs=127.19.52.126:45755
--builtin_ntp_servers=127.19.52.84:37205
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260812 06:59:17.043020 9906 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:59:17.043367 9906 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:59:17.043444 9906 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 06:59:17.047668 9906 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 06:59:17.047778 9906 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 06:59:17.047881 9906 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.52.65
I20260812 06:59:17.052947 9906 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.52.84:37205
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786517687384214-19665-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786517687384214-19665-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.52.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786517687384214-19665-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.19.52.65
--webserver_port=0
--tserver_master_addrs=127.19.52.126:45755
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9906
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.52.65
--log_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786517687384214-19665-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 06:15:54 UTC on ccycloud.quasar-rnfzce.root.comops.site
build id 4581989
I20260812 06:59:17.054155 9906 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:59:17.055593 9906 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 06:59:17.065663 9915 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:59:17.065663 9911 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:59:17.065876 9906 server_base.cc:1034] running on GCE node
W20260812 06:59:17.065701 9912 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:59:17.066419 9906 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:59:17.067183 9906 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:59:17.068382 9906 hybrid_clock.cc:648] HybridClock initialized: now 1786517957068363 us; error 34 us; skew 500 ppm
I20260812 06:59:17.070526 9906 webserver.cc:460] Webserver started at http://127.19.52.65:42507/ using document root <none> and password file <none>
I20260812 06:59:17.071143 9906 fs_manager.cc:359] Metadata directory not provided
I20260812 06:59:17.071242 9906 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:59:17.071485 9906 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 06:59:17.073906 9906 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786517687384214-19665-0/minicluster-data/ts-0/data/instance:
uuid: "03563d6c67c147a6896392f09207b598"
format_stamp: "Formatted at 2026-08-12 06:59:17 on dist-test-slave-65mx"
server_key: "85ee386402df1400d3c98f2dc30a28f0"
server_key_iv: "4b79c6aed59f103cfcd1c04618114009"
server_key_version: "encryptionkey@0"
I20260812 06:59:17.074536 9906 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786517687384214-19665-0/minicluster-data/ts-0/wal/instance:
uuid: "03563d6c67c147a6896392f09207b598"
format_stamp: "Formatted at 2026-08-12 06:59:17 on dist-test-slave-65mx"
server_key: "85ee386402df1400d3c98f2dc30a28f0"
server_key_iv: "4b79c6aed59f103cfcd1c04618114009"
server_key_version: "encryptionkey@0"
I20260812 06:59:17.078722 9906 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260812 06:59:17.081569 9921 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 06:59:17.083074 9906 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 06:59:17.083220 9906 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786517687384214-19665-0/minicluster-data/ts-0/data,/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786517687384214-19665-0/minicluster-data/ts-0/wal
uuid: "03563d6c67c147a6896392f09207b598"
format_stamp: "Formatted at 2026-08-12 06:59:17 on dist-test-slave-65mx"
server_key: "85ee386402df1400d3c98f2dc30a28f0"
server_key_iv: "4b79c6aed59f103cfcd1c04618114009"
server_key_version: "encryptionkey@0"
I20260812 06:59:17.083359 9906 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786517687384214-19665-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786517687384214-19665-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786517687384214-19665-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:59:17.111686 9906 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 06:59:17.112485 9906 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 06:59:17.112771 9906 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:59:17.113502 9906 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 06:59:17.114555 9906 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 06:59:17.114629 9906 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 06:59:17.114701 9906 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 06:59:17.114745 9906 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 06:59:17.124782 9906 rpc_server.cc:307] RPC server started. Bound to: 127.19.52.65:35197
I20260812 06:59:17.124806 10053 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.52.65:35197 every 8 connection(s)
I20260812 06:59:17.125741 9906 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786517687384214-19665-0/minicluster-data/ts-0/data/info.pb
I20260812 06:59:17.133400 19665 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu as pid 9906
I20260812 06:59:17.133570 19665 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786517687384214-19665-0/minicluster-data/ts-0/wal/instance
I20260812 06:59:17.133908 19665 external_mini_cluster.cc:1428] Setting key afc4124e28f53e2af9e3a507e92002da
I20260812 06:59:17.136355 19665 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu
/tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786517687384214-19665-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786517687384214-19665-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786517687384214-19665-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786517687384214-19665-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.52.66:0
--local_ip_for_outbound_sockets=127.19.52.66
--webserver_interface=127.19.52.66
--webserver_port=0
--tserver_master_addrs=127.19.52.126:45755
--builtin_ntp_servers=127.19.52.84:37205
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260812 06:59:17.136523 9879 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:59:17.127565 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.52.65:59973 (local address 127.19.52.126:45755)
0812 06:59:17.127711 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0812 06:59:17.127715 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 06:59:17.128776 (+ 1061us) server_negotiation.cc:407] Connection header received
0812 06:59:17.130618 (+ 1842us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:59:17.130621 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:59:17.130684 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:59:17.130759 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:59:17.131713 (+ 954us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:59:17.132480 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:59:17.134125 (+ 1645us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:59:17.134464 (+ 339us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:59:17.134620 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:59:17.135920 (+ 1300us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:59:17.135924 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:59:17.135927 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 06:59:17.135941 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:59:17.135964 (+ 23us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 06:59:17.135974 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:59:17.136061 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0812 06:59:17.136260 (+ 199us) server_negotiation.cc:299] Negotiation successful
0812 06:59:17.136334 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260812 06:59:17.137677 10057 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:59:17.127892 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.126:45755 (local address 127.19.52.65:59973)
0812 06:59:17.128614 (+ 722us) negotiation.cc:107] Waiting for socket to connect
0812 06:59:17.128673 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0812 06:59:17.130282 (+ 1609us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:59:17.130931 (+ 649us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:59:17.130939 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:59:17.131029 (+ 90us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:59:17.131561 (+ 532us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:59:17.131575 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:59:17.132676 (+ 1101us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:59:17.132679 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:59:17.133936 (+ 1257us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:59:17.133951 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:59:17.134689 (+ 738us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:59:17.134694 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:59:17.134894 (+ 200us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:59:17.135612 (+ 718us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 06:59:17.135615 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:59:17.135674 (+ 59us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 06:59:17.135759 (+ 85us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 06:59:17.136092 (+ 333us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 06:59:17.136102 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 06:59:17.136107 (+ 5us) client_negotiation.cc:769] Sending connection context
0812 06:59:17.136230 (+ 123us) client_negotiation.cc:240] Negotiation successful
0812 06:59:17.136745 (+ 515us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":267,"thread_start_us":121,"threads_started":1}
I20260812 06:59:17.139052 10055 heartbeater.cc:347] Connected to a master server at 127.19.52.126:45755
I20260812 06:59:17.139351 10055 heartbeater.cc:464] Registering TS with master...
I20260812 06:59:17.139917 10055 heartbeater.cc:511] Master 127.19.52.126:45755 requested a full tablet report, sending...
I20260812 06:59:17.141463 9837 ts_manager.cc:194] Registered new tserver with Master: 03563d6c67c147a6896392f09207b598 (127.19.52.65:35197)
I20260812 06:59:17.142716 9837 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.19.52.65:59973
W20260812 06:59:17.254094 10058 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:59:17.254439 10058 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:59:17.254523 10058 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 06:59:17.258692 10058 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 06:59:17.258800 10058 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 06:59:17.258898 10058 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.52.66
I20260812 06:59:17.263823 10058 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.52.84:37205
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786517687384214-19665-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786517687384214-19665-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.52.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786517687384214-19665-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.19.52.66
--webserver_port=0
--tserver_master_addrs=127.19.52.126:45755
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10058
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.52.66
--log_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786517687384214-19665-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 06:15:54 UTC on ccycloud.quasar-rnfzce.root.comops.site
build id 4581989
I20260812 06:59:17.265053 10058 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:59:17.266461 10058 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 06:59:17.276387 10065 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:59:17.276403 10058 server_base.cc:1034] running on GCE node
W20260812 06:59:17.276546 10064 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:59:17.276719 10067 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:59:17.277216 10058 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:59:17.278007 10058 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:59:17.279230 10058 hybrid_clock.cc:648] HybridClock initialized: now 1786517957279192 us; error 50 us; skew 500 ppm
I20260812 06:59:17.281257 10058 webserver.cc:460] Webserver started at http://127.19.52.66:39409/ using document root <none> and password file <none>
I20260812 06:59:17.281860 10058 fs_manager.cc:359] Metadata directory not provided
I20260812 06:59:17.281957 10058 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:59:17.282204 10058 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 06:59:17.284341 10058 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786517687384214-19665-0/minicluster-data/ts-1/data/instance:
uuid: "07bc89147f6743668cd971afde67c3b9"
format_stamp: "Formatted at 2026-08-12 06:59:17 on dist-test-slave-65mx"
server_key: "0907127922aba4055599c85b4e9a9a00"
server_key_iv: "05d31a59da778580a77f1fb0881fc868"
server_key_version: "encryptionkey@0"
I20260812 06:59:17.284976 10058 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786517687384214-19665-0/minicluster-data/ts-1/wal/instance:
uuid: "07bc89147f6743668cd971afde67c3b9"
format_stamp: "Formatted at 2026-08-12 06:59:17 on dist-test-slave-65mx"
server_key: "0907127922aba4055599c85b4e9a9a00"
server_key_iv: "05d31a59da778580a77f1fb0881fc868"
server_key_version: "encryptionkey@0"
I20260812 06:59:17.289091 10058 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260812 06:59:17.291785 10075 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 06:59:17.293165 10058 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 06:59:17.293308 10058 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786517687384214-19665-0/minicluster-data/ts-1/data,/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786517687384214-19665-0/minicluster-data/ts-1/wal
uuid: "07bc89147f6743668cd971afde67c3b9"
format_stamp: "Formatted at 2026-08-12 06:59:17 on dist-test-slave-65mx"
server_key: "0907127922aba4055599c85b4e9a9a00"
server_key_iv: "05d31a59da778580a77f1fb0881fc868"
server_key_version: "encryptionkey@0"
I20260812 06:59:17.293438 10058 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786517687384214-19665-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786517687384214-19665-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786517687384214-19665-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:59:17.305668 10058 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 06:59:17.306385 10058 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 06:59:17.306600 10058 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:59:17.307214 10058 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 06:59:17.308166 10058 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 06:59:17.308239 10058 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 06:59:17.308314 10058 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 06:59:17.308351 10058 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 06:59:17.317416 10058 rpc_server.cc:307] RPC server started. Bound to: 127.19.52.66:36125
I20260812 06:59:17.317451 10210 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.52.66:36125 every 8 connection(s)
I20260812 06:59:17.318351 10058 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786517687384214-19665-0/minicluster-data/ts-1/data/info.pb
I20260812 06:59:17.325866 19665 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu as pid 10058
I20260812 06:59:17.326068 19665 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786517687384214-19665-0/minicluster-data/ts-1/wal/instance
I20260812 06:59:17.326411 19665 external_mini_cluster.cc:1428] Setting key 232d385308818e2f7fb3e27164b0b02a
I20260812 06:59:17.328332 9879 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:59:17.320344 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.52.66:43239 (local address 127.19.52.126:45755)
0812 06:59:17.320623 (+ 279us) server_negotiation.cc:206] Beginning negotiation
0812 06:59:17.320636 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0812 06:59:17.321684 (+ 1048us) server_negotiation.cc:407] Connection header received
0812 06:59:17.323119 (+ 1435us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:59:17.323122 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:59:17.323174 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:59:17.323256 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:59:17.324169 (+ 913us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:59:17.324758 (+ 589us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:59:17.326009 (+ 1251us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:59:17.326355 (+ 346us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:59:17.326497 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:59:17.327803 (+ 1306us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:59:17.327806 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:59:17.327809 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 06:59:17.327823 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:59:17.327853 (+ 30us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 06:59:17.327881 (+ 28us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:59:17.327958 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0812 06:59:17.328168 (+ 210us) server_negotiation.cc:299] Negotiation successful
0812 06:59:17.328223 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260812 06:59:17.328727 19665 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu
/tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786517687384214-19665-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786517687384214-19665-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786517687384214-19665-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786517687384214-19665-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.52.67:0
--local_ip_for_outbound_sockets=127.19.52.67
--webserver_interface=127.19.52.67
--webserver_port=0
--tserver_master_addrs=127.19.52.126:45755
--builtin_ntp_servers=127.19.52.84:37205
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260812 06:59:17.329856 10213 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:59:17.320822 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.126:45755 (local address 127.19.52.66:43239)
0812 06:59:17.321529 (+ 707us) negotiation.cc:107] Waiting for socket to connect
0812 06:59:17.321572 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0812 06:59:17.322896 (+ 1324us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:59:17.323429 (+ 533us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:59:17.323436 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:59:17.323520 (+ 84us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:59:17.324021 (+ 501us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:59:17.324033 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:59:17.324906 (+ 873us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:59:17.324909 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:59:17.325853 (+ 944us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:59:17.325863 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:59:17.326641 (+ 778us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:59:17.326647 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:59:17.326808 (+ 161us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:59:17.327514 (+ 706us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 06:59:17.327515 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:59:17.327558 (+ 43us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 06:59:17.327629 (+ 71us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 06:59:17.327982 (+ 353us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 06:59:17.327993 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 06:59:17.328000 (+ 7us) client_negotiation.cc:769] Sending connection context
0812 06:59:17.328153 (+ 153us) client_negotiation.cc:240] Negotiation successful
0812 06:59:17.328482 (+ 329us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":298,"thread_start_us":136,"threads_started":1}
I20260812 06:59:17.331075 10211 heartbeater.cc:347] Connected to a master server at 127.19.52.126:45755
I20260812 06:59:17.331324 10211 heartbeater.cc:464] Registering TS with master...
I20260812 06:59:17.331882 10211 heartbeater.cc:511] Master 127.19.52.126:45755 requested a full tablet report, sending...
I20260812 06:59:17.333048 9837 ts_manager.cc:194] Registered new tserver with Master: 07bc89147f6743668cd971afde67c3b9 (127.19.52.66:36125)
I20260812 06:59:17.333655 9837 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.19.52.66:43239
W20260812 06:59:17.458050 10214 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:59:17.458424 10214 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:59:17.458509 10214 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 06:59:17.463019 10214 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 06:59:17.463117 10214 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 06:59:17.463215 10214 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.52.67
I20260812 06:59:17.468297 10214 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.52.84:37205
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786517687384214-19665-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786517687384214-19665-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.52.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786517687384214-19665-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.19.52.67
--webserver_port=0
--tserver_master_addrs=127.19.52.126:45755
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10214
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.52.67
--log_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786517687384214-19665-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 06:15:54 UTC on ccycloud.quasar-rnfzce.root.comops.site
build id 4581989
I20260812 06:59:17.469523 10214 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:59:17.470976 10214 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 06:59:17.481096 10219 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:59:17.481122 10221 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:59:17.481178 10214 server_base.cc:1034] running on GCE node
W20260812 06:59:17.481096 10224 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:59:17.481822 10214 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:59:17.482496 10214 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:59:17.483711 10214 hybrid_clock.cc:648] HybridClock initialized: now 1786517957483694 us; error 28 us; skew 500 ppm
I20260812 06:59:17.485735 10214 webserver.cc:460] Webserver started at http://127.19.52.67:45719/ using document root <none> and password file <none>
I20260812 06:59:17.486377 10214 fs_manager.cc:359] Metadata directory not provided
I20260812 06:59:17.486491 10214 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:59:17.486747 10214 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 06:59:17.489060 10214 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786517687384214-19665-0/minicluster-data/ts-2/data/instance:
uuid: "dd60837110e8419f8feac79220e0508c"
format_stamp: "Formatted at 2026-08-12 06:59:17 on dist-test-slave-65mx"
server_key: "1518981f91a1304085acbdb283ebd932"
server_key_iv: "826b913f1644b10a7796f4525832d6ac"
server_key_version: "encryptionkey@0"
I20260812 06:59:17.489678 10214 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786517687384214-19665-0/minicluster-data/ts-2/wal/instance:
uuid: "dd60837110e8419f8feac79220e0508c"
format_stamp: "Formatted at 2026-08-12 06:59:17 on dist-test-slave-65mx"
server_key: "1518981f91a1304085acbdb283ebd932"
server_key_iv: "826b913f1644b10a7796f4525832d6ac"
server_key_version: "encryptionkey@0"
I20260812 06:59:17.493718 10214 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.001s
I20260812 06:59:17.496523 10231 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 06:59:17.498200 10214 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260812 06:59:17.498315 10214 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786517687384214-19665-0/minicluster-data/ts-2/data,/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786517687384214-19665-0/minicluster-data/ts-2/wal
uuid: "dd60837110e8419f8feac79220e0508c"
format_stamp: "Formatted at 2026-08-12 06:59:17 on dist-test-slave-65mx"
server_key: "1518981f91a1304085acbdb283ebd932"
server_key_iv: "826b913f1644b10a7796f4525832d6ac"
server_key_version: "encryptionkey@0"
I20260812 06:59:17.498425 10214 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786517687384214-19665-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786517687384214-19665-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786517687384214-19665-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:59:17.512773 10214 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 06:59:17.513466 10214 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 06:59:17.513629 10214 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:59:17.514206 10214 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 06:59:17.515195 10214 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 06:59:17.515246 10214 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 06:59:17.515288 10214 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 06:59:17.515304 10214 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 06:59:17.525024 10214 rpc_server.cc:307] RPC server started. Bound to: 127.19.52.67:42151
I20260812 06:59:17.525053 10359 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.52.67:42151 every 8 connection(s)
I20260812 06:59:17.526041 10214 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786517687384214-19665-0/minicluster-data/ts-2/data/info.pb
I20260812 06:59:17.527287 19665 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu as pid 10214
I20260812 06:59:17.527407 19665 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786517687384214-19665-0/minicluster-data/ts-2/wal/instance
I20260812 06:59:17.527747 19665 external_mini_cluster.cc:1428] Setting key 3f32b235bb8b1a6aaf869798a9c1f318
I20260812 06:59:17.536298 9879 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:59:17.528225 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.52.67:40537 (local address 127.19.52.126:45755)
0812 06:59:17.528370 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0812 06:59:17.528374 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 06:59:17.529782 (+ 1408us) server_negotiation.cc:407] Connection header received
0812 06:59:17.531183 (+ 1401us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:59:17.531186 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:59:17.531233 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:59:17.531307 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:59:17.532141 (+ 834us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:59:17.532812 (+ 671us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:59:17.534026 (+ 1214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:59:17.534244 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:59:17.534325 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:59:17.535875 (+ 1550us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:59:17.535877 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:59:17.535879 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 06:59:17.535893 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:59:17.535911 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 06:59:17.535919 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:59:17.535985 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0812 06:59:17.536124 (+ 139us) server_negotiation.cc:299] Negotiation successful
0812 06:59:17.536179 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260812 06:59:17.537323 10364 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:59:17.528776 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.126:45755 (local address 127.19.52.67:40537)
0812 06:59:17.529577 (+ 801us) negotiation.cc:107] Waiting for socket to connect
0812 06:59:17.529646 (+ 69us) client_negotiation.cc:174] Beginning negotiation
0812 06:59:17.530987 (+ 1341us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:59:17.531433 (+ 446us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:59:17.531439 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:59:17.531520 (+ 81us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:59:17.532013 (+ 493us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:59:17.532025 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:59:17.532943 (+ 918us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:59:17.532946 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:59:17.533871 (+ 925us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:59:17.533881 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:59:17.534590 (+ 709us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:59:17.534599 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:59:17.534763 (+ 164us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:59:17.535568 (+ 805us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 06:59:17.535569 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:59:17.535612 (+ 43us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 06:59:17.535713 (+ 101us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 06:59:17.535999 (+ 286us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 06:59:17.536007 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 06:59:17.536011 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 06:59:17.536109 (+ 98us) client_negotiation.cc:240] Negotiation successful
0812 06:59:17.536387 (+ 278us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":327,"thread_start_us":153,"threads_started":1}
I20260812 06:59:17.538594 10361 heartbeater.cc:347] Connected to a master server at 127.19.52.126:45755
I20260812 06:59:17.538853 10361 heartbeater.cc:464] Registering TS with master...
I20260812 06:59:17.539403 10361 heartbeater.cc:511] Master 127.19.52.126:45755 requested a full tablet report, sending...
I20260812 06:59:17.540868 9837 ts_manager.cc:194] Registered new tserver with Master: dd60837110e8419f8feac79220e0508c (127.19.52.67:42151)
I20260812 06:59:17.541648 9837 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.19.52.67:40537
I20260812 06:59:17.542183 19665 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 06:59:17.548252 9879 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:59:17.544300 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34680 (local address 127.19.52.126:45755)
0812 06:59:17.544463 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0812 06:59:17.544468 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 06:59:17.544637 (+ 169us) server_negotiation.cc:407] Connection header received
0812 06:59:17.544799 (+ 162us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:59:17.544803 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:59:17.544853 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:59:17.544940 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:59:17.545253 (+ 313us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:59:17.545758 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:59:17.546608 (+ 850us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:59:17.546829 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:59:17.546942 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:59:17.547799 (+ 857us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:59:17.547803 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:59:17.547806 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 06:59:17.547826 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:59:17.547844 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 06:59:17.547851 (+ 7us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:59:17.547933 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0812 06:59:17.548028 (+ 95us) server_negotiation.cc:299] Negotiation successful
0812 06:59:17.548086 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260812 06:59:17.551481 9837 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:34680:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 06:59:17.554780 9837 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 06:59:17.571549 10380 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:59:17.565640 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53390 (local address 127.19.52.67:42151)
0812 06:59:17.566647 (+ 1007us) server_negotiation.cc:206] Beginning negotiation
0812 06:59:17.566655 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 06:59:17.566673 (+ 18us) server_negotiation.cc:407] Connection header received
0812 06:59:17.566733 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:59:17.566737 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:59:17.566861 (+ 124us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:59:17.567027 (+ 166us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 06:59:17.567270 (+ 243us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:59:17.568458 (+ 1188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:59:17.570153 (+ 1695us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:59:17.570710 (+ 557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:59:17.570848 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:59:17.571076 (+ 228us) server_negotiation.cc:1035] Waiting for connection context
0812 06:59:17.571252 (+ 176us) server_negotiation.cc:299] Negotiation successful
0812 06:59:17.571365 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":921,"thread_start_us":137,"threads_started":1}
I20260812 06:59:17.571543 10378 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:59:17.565102 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.67:42151 (local address 127.0.0.1:53390)
0812 06:59:17.565684 (+ 582us) negotiation.cc:107] Waiting for socket to connect
0812 06:59:17.565698 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 06:59:17.565792 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:59:17.566998 (+ 1206us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:59:17.567001 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:59:17.567020 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 06:59:17.567129 (+ 109us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:59:17.567139 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:59:17.568612 (+ 1473us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:59:17.568615 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:59:17.569979 (+ 1364us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:59:17.570016 (+ 37us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:59:17.570852 (+ 836us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:59:17.570856 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:59:17.571024 (+ 168us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:59:17.571071 (+ 47us) client_negotiation.cc:769] Sending connection context
0812 06:59:17.571195 (+ 124us) client_negotiation.cc:240] Negotiation successful
0812 06:59:17.571367 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":517,"mutex_wait_us":395,"thread_start_us":100,"threads_started":1}
I20260812 06:59:17.571547 10379 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:59:17.565550 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.66:36125 (local address 127.0.0.1:49844)
0812 06:59:17.566316 (+ 766us) negotiation.cc:107] Waiting for socket to connect
0812 06:59:17.566328 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 06:59:17.566415 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:59:17.567184 (+ 769us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:59:17.567188 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:59:17.567210 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 06:59:17.567299 (+ 89us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:59:17.567303 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:59:17.568280 (+ 977us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:59:17.568286 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:59:17.569922 (+ 1636us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:59:17.569934 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:59:17.570855 (+ 921us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:59:17.570859 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:59:17.571027 (+ 168us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:59:17.571072 (+ 45us) client_negotiation.cc:769] Sending connection context
0812 06:59:17.571220 (+ 148us) client_negotiation.cc:240] Negotiation successful
0812 06:59:17.571360 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":704,"mutex_wait_us":128,"thread_start_us":100,"threads_started":1}
I20260812 06:59:17.571885 10381 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:59:17.566014 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49844 (local address 127.19.52.66:36125)
0812 06:59:17.566551 (+ 537us) server_negotiation.cc:206] Beginning negotiation
0812 06:59:17.566576 (+ 25us) server_negotiation.cc:399] Waiting for connection header
0812 06:59:17.566608 (+ 32us) server_negotiation.cc:407] Connection header received
0812 06:59:17.566685 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:59:17.566690 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:59:17.566866 (+ 176us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:59:17.567037 (+ 171us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 06:59:17.567414 (+ 377us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:59:17.568158 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:59:17.570083 (+ 1925us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:59:17.570710 (+ 627us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:59:17.570846 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:59:17.571072 (+ 226us) server_negotiation.cc:1035] Waiting for connection context
0812 06:59:17.571621 (+ 549us) server_negotiation.cc:299] Negotiation successful
0812 06:59:17.571737 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":347,"thread_start_us":147,"threads_started":1}
I20260812 06:59:17.573122 10376 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:59:17.564823 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.65:35197 (local address 127.0.0.1:57586)
0812 06:59:17.565182 (+ 359us) negotiation.cc:107] Waiting for socket to connect
0812 06:59:17.565234 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0812 06:59:17.565364 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:59:17.566788 (+ 1424us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:59:17.566795 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:59:17.566818 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 06:59:17.566976 (+ 158us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:59:17.566986 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:59:17.568612 (+ 1626us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:59:17.568615 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:59:17.569926 (+ 1311us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:59:17.569934 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:59:17.572686 (+ 2752us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:59:17.572690 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:59:17.572788 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:59:17.572821 (+ 33us) client_negotiation.cc:769] Sending connection context
0812 06:59:17.572894 (+ 73us) client_negotiation.cc:240] Negotiation successful
0812 06:59:17.572985 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":179,"mutex_wait_us":133,"thread_start_us":82,"threads_started":1}
I20260812 06:59:17.573429 10377 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:59:17.564970 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57586 (local address 127.19.52.65:35197)
0812 06:59:17.565962 (+ 992us) server_negotiation.cc:206] Beginning negotiation
0812 06:59:17.565976 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0812 06:59:17.566004 (+ 28us) server_negotiation.cc:407] Connection header received
0812 06:59:17.566083 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:59:17.566089 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:59:17.566316 (+ 227us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:59:17.566577 (+ 261us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 06:59:17.567116 (+ 539us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:59:17.568435 (+ 1319us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:59:17.571148 (+ 2713us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:59:17.572526 (+ 1378us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:59:17.572676 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:59:17.573108 (+ 432us) server_negotiation.cc:1035] Waiting for connection context
0812 06:59:17.573166 (+ 58us) server_negotiation.cc:299] Negotiation successful
0812 06:59:17.573265 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":879,"thread_start_us":180,"threads_started":1}
I20260812 06:59:17.575567 10289 tablet_service.cc:1467] Processing CreateTablet for tablet 3c2f304ec35e4bdbb209f5bf1c1c2b62 (DEFAULT_TABLE table=test-table [id=6e17801c7faa4226bc3242471f123963]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 06:59:17.575899 10132 tablet_service.cc:1467] Processing CreateTablet for tablet 3c2f304ec35e4bdbb209f5bf1c1c2b62 (DEFAULT_TABLE table=test-table [id=6e17801c7faa4226bc3242471f123963]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 06:59:17.576352 9979 tablet_service.cc:1467] Processing CreateTablet for tablet 3c2f304ec35e4bdbb209f5bf1c1c2b62 (DEFAULT_TABLE table=test-table [id=6e17801c7faa4226bc3242471f123963]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 06:59:17.577023 10132 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3c2f304ec35e4bdbb209f5bf1c1c2b62. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 06:59:17.577067 10289 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3c2f304ec35e4bdbb209f5bf1c1c2b62. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 06:59:17.577383 9979 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3c2f304ec35e4bdbb209f5bf1c1c2b62. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 06:59:17.583880 10383 tablet_bootstrap.cc:492] T 3c2f304ec35e4bdbb209f5bf1c1c2b62 P 03563d6c67c147a6896392f09207b598: Bootstrap starting.
I20260812 06:59:17.585559 10384 tablet_bootstrap.cc:492] T 3c2f304ec35e4bdbb209f5bf1c1c2b62 P 07bc89147f6743668cd971afde67c3b9: Bootstrap starting.
I20260812 06:59:17.586787 10383 tablet_bootstrap.cc:654] T 3c2f304ec35e4bdbb209f5bf1c1c2b62 P 03563d6c67c147a6896392f09207b598: Neither blocks nor log segments found. Creating new log.
I20260812 06:59:17.587725 10383 log.cc:824] T 3c2f304ec35e4bdbb209f5bf1c1c2b62 P 03563d6c67c147a6896392f09207b598: Log is configured to *not* fsync() on all Append() calls
I20260812 06:59:17.587967 10384 tablet_bootstrap.cc:654] T 3c2f304ec35e4bdbb209f5bf1c1c2b62 P 07bc89147f6743668cd971afde67c3b9: Neither blocks nor log segments found. Creating new log.
I20260812 06:59:17.588603 10385 tablet_bootstrap.cc:492] T 3c2f304ec35e4bdbb209f5bf1c1c2b62 P dd60837110e8419f8feac79220e0508c: Bootstrap starting.
I20260812 06:59:17.588894 10384 log.cc:824] T 3c2f304ec35e4bdbb209f5bf1c1c2b62 P 07bc89147f6743668cd971afde67c3b9: Log is configured to *not* fsync() on all Append() calls
I20260812 06:59:17.590726 10384 tablet_bootstrap.cc:492] T 3c2f304ec35e4bdbb209f5bf1c1c2b62 P 07bc89147f6743668cd971afde67c3b9: No bootstrap required, opened a new log
I20260812 06:59:17.590809 10383 tablet_bootstrap.cc:492] T 3c2f304ec35e4bdbb209f5bf1c1c2b62 P 03563d6c67c147a6896392f09207b598: No bootstrap required, opened a new log
I20260812 06:59:17.590883 10385 tablet_bootstrap.cc:654] T 3c2f304ec35e4bdbb209f5bf1c1c2b62 P dd60837110e8419f8feac79220e0508c: Neither blocks nor log segments found. Creating new log.
I20260812 06:59:17.590971 10384 ts_tablet_manager.cc:1397] T 3c2f304ec35e4bdbb209f5bf1c1c2b62 P 07bc89147f6743668cd971afde67c3b9: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260812 06:59:17.591038 10383 ts_tablet_manager.cc:1397] T 3c2f304ec35e4bdbb209f5bf1c1c2b62 P 03563d6c67c147a6896392f09207b598: Time spent bootstrapping tablet: real 0.007s user 0.004s sys 0.000s
I20260812 06:59:17.591820 10385 log.cc:824] T 3c2f304ec35e4bdbb209f5bf1c1c2b62 P dd60837110e8419f8feac79220e0508c: Log is configured to *not* fsync() on all Append() calls
I20260812 06:59:17.594487 10385 tablet_bootstrap.cc:492] T 3c2f304ec35e4bdbb209f5bf1c1c2b62 P dd60837110e8419f8feac79220e0508c: No bootstrap required, opened a new log
I20260812 06:59:17.594720 10385 ts_tablet_manager.cc:1397] T 3c2f304ec35e4bdbb209f5bf1c1c2b62 P dd60837110e8419f8feac79220e0508c: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260812 06:59:17.595136 10384 raft_consensus.cc:348] T 3c2f304ec35e4bdbb209f5bf1c1c2b62 P 07bc89147f6743668cd971afde67c3b9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03563d6c67c147a6896392f09207b598" member_type: VOTER last_known_addr { host: "127.19.52.65" port: 35197 } } peers { permanent_uuid: "07bc89147f6743668cd971afde67c3b9" member_type: VOTER last_known_addr { host: "127.19.52.66" port: 36125 } } peers { permanent_uuid: "dd60837110e8419f8feac79220e0508c" member_type: VOTER last_known_addr { host: "127.19.52.67" port: 42151 } }
I20260812 06:59:17.595475 10384 raft_consensus.cc:374] T 3c2f304ec35e4bdbb209f5bf1c1c2b62 P 07bc89147f6743668cd971afde67c3b9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 06:59:17.595536 10384 raft_consensus.cc:729] T 3c2f304ec35e4bdbb209f5bf1c1c2b62 P 07bc89147f6743668cd971afde67c3b9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 07bc89147f6743668cd971afde67c3b9, State: Initialized, Role: FOLLOWER
I20260812 06:59:17.595592 10383 raft_consensus.cc:348] T 3c2f304ec35e4bdbb209f5bf1c1c2b62 P 03563d6c67c147a6896392f09207b598 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03563d6c67c147a6896392f09207b598" member_type: VOTER last_known_addr { host: "127.19.52.65" port: 35197 } } peers { permanent_uuid: "07bc89147f6743668cd971afde67c3b9" member_type: VOTER last_known_addr { host: "127.19.52.66" port: 36125 } } peers { permanent_uuid: "dd60837110e8419f8feac79220e0508c" member_type: VOTER last_known_addr { host: "127.19.52.67" port: 42151 } }
I20260812 06:59:17.595857 10383 raft_consensus.cc:374] T 3c2f304ec35e4bdbb209f5bf1c1c2b62 P 03563d6c67c147a6896392f09207b598 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 06:59:17.595921 10383 raft_consensus.cc:729] T 3c2f304ec35e4bdbb209f5bf1c1c2b62 P 03563d6c67c147a6896392f09207b598 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 03563d6c67c147a6896392f09207b598, State: Initialized, Role: FOLLOWER
I20260812 06:59:17.596069 10384 consensus_queue.cc:260] T 3c2f304ec35e4bdbb209f5bf1c1c2b62 P 07bc89147f6743668cd971afde67c3b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03563d6c67c147a6896392f09207b598" member_type: VOTER last_known_addr { host: "127.19.52.65" port: 35197 } } peers { permanent_uuid: "07bc89147f6743668cd971afde67c3b9" member_type: VOTER last_known_addr { host: "127.19.52.66" port: 36125 } } peers { permanent_uuid: "dd60837110e8419f8feac79220e0508c" member_type: VOTER last_known_addr { host: "127.19.52.67" port: 42151 } }
I20260812 06:59:17.596511 10383 consensus_queue.cc:260] T 3c2f304ec35e4bdbb209f5bf1c1c2b62 P 03563d6c67c147a6896392f09207b598 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03563d6c67c147a6896392f09207b598" member_type: VOTER last_known_addr { host: "127.19.52.65" port: 35197 } } peers { permanent_uuid: "07bc89147f6743668cd971afde67c3b9" member_type: VOTER last_known_addr { host: "127.19.52.66" port: 36125 } } peers { permanent_uuid: "dd60837110e8419f8feac79220e0508c" member_type: VOTER last_known_addr { host: "127.19.52.67" port: 42151 } }
I20260812 06:59:17.596834 10211 heartbeater.cc:503] Master 127.19.52.126:45755 was elected leader, sending a full tablet report...
I20260812 06:59:17.597280 10384 ts_tablet_manager.cc:1428] T 3c2f304ec35e4bdbb209f5bf1c1c2b62 P 07bc89147f6743668cd971afde67c3b9: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260812 06:59:17.597586 10383 ts_tablet_manager.cc:1428] T 3c2f304ec35e4bdbb209f5bf1c1c2b62 P 03563d6c67c147a6896392f09207b598: Time spent starting tablet: real 0.006s user 0.008s sys 0.000s
I20260812 06:59:17.597925 10055 heartbeater.cc:503] Master 127.19.52.126:45755 was elected leader, sending a full tablet report...
I20260812 06:59:17.599324 10385 raft_consensus.cc:348] T 3c2f304ec35e4bdbb209f5bf1c1c2b62 P dd60837110e8419f8feac79220e0508c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03563d6c67c147a6896392f09207b598" member_type: VOTER last_known_addr { host: "127.19.52.65" port: 35197 } } peers { permanent_uuid: "07bc89147f6743668cd971afde67c3b9" member_type: VOTER last_known_addr { host: "127.19.52.66" port: 36125 } } peers { permanent_uuid: "dd60837110e8419f8feac79220e0508c" member_type: VOTER last_known_addr { host: "127.19.52.67" port: 42151 } }
I20260812 06:59:17.599615 10385 raft_consensus.cc:374] T 3c2f304ec35e4bdbb209f5bf1c1c2b62 P dd60837110e8419f8feac79220e0508c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 06:59:17.599675 10385 raft_consensus.cc:729] T 3c2f304ec35e4bdbb209f5bf1c1c2b62 P dd60837110e8419f8feac79220e0508c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dd60837110e8419f8feac79220e0508c, State: Initialized, Role: FOLLOWER
I20260812 06:59:17.600149 10385 consensus_queue.cc:260] T 3c2f304ec35e4bdbb209f5bf1c1c2b62 P dd60837110e8419f8feac79220e0508c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03563d6c67c147a6896392f09207b598" member_type: VOTER last_known_addr { host: "127.19.52.65" port: 35197 } } peers { permanent_uuid: "07bc89147f6743668cd971afde67c3b9" member_type: VOTER last_known_addr { host: "127.19.52.66" port: 36125 } } peers { permanent_uuid: "dd60837110e8419f8feac79220e0508c" member_type: VOTER last_known_addr { host: "127.19.52.67" port: 42151 } }
I20260812 06:59:17.601235 10385 ts_tablet_manager.cc:1428] T 3c2f304ec35e4bdbb209f5bf1c1c2b62 P dd60837110e8419f8feac79220e0508c: Time spent starting tablet: real 0.006s user 0.008s sys 0.000s
I20260812 06:59:17.601382 10361 heartbeater.cc:503] Master 127.19.52.126:45755 was elected leader, sending a full tablet report...
W20260812 06:59:17.627027 10056 tablet.cc:2369] T 3c2f304ec35e4bdbb209f5bf1c1c2b62 P 03563d6c67c147a6896392f09207b598: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 06:59:17.778754 10363 tablet.cc:2369] T 3c2f304ec35e4bdbb209f5bf1c1c2b62 P dd60837110e8419f8feac79220e0508c: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 06:59:17.803179 10390 raft_consensus.cc:482] T 3c2f304ec35e4bdbb209f5bf1c1c2b62 P 03563d6c67c147a6896392f09207b598 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 06:59:17.803447 10390 raft_consensus.cc:504] T 3c2f304ec35e4bdbb209f5bf1c1c2b62 P 03563d6c67c147a6896392f09207b598 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03563d6c67c147a6896392f09207b598" member_type: VOTER last_known_addr { host: "127.19.52.65" port: 35197 } } peers { permanent_uuid: "07bc89147f6743668cd971afde67c3b9" member_type: VOTER last_known_addr { host: "127.19.52.66" port: 36125 } } peers { permanent_uuid: "dd60837110e8419f8feac79220e0508c" member_type: VOTER last_known_addr { host: "127.19.52.67" port: 42151 } }
I20260812 06:59:17.804590 10390 leader_election.cc:288] T 3c2f304ec35e4bdbb209f5bf1c1c2b62 P 03563d6c67c147a6896392f09207b598 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 07bc89147f6743668cd971afde67c3b9 (127.19.52.66:36125), dd60837110e8419f8feac79220e0508c (127.19.52.67:42151)
I20260812 06:59:17.808661 10381 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:59:17.804979 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.52.65:51143 (local address 127.19.52.66:36125)
0812 06:59:17.805207 (+ 228us) server_negotiation.cc:206] Beginning negotiation
0812 06:59:17.805212 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 06:59:17.805361 (+ 149us) server_negotiation.cc:407] Connection header received
0812 06:59:17.805545 (+ 184us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:59:17.805548 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:59:17.805602 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:59:17.805684 (+ 82us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 06:59:17.805962 (+ 278us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:59:17.806500 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:59:17.807601 (+ 1101us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:59:17.807998 (+ 397us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:59:17.808152 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:59:17.808207 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0812 06:59:17.808423 (+ 216us) server_negotiation.cc:299] Negotiation successful
0812 06:59:17.808497 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":142}
I20260812 06:59:17.808701 10393 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:59:17.804889 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.66:36125 (local address 127.19.52.65:51143)
0812 06:59:17.805269 (+ 380us) negotiation.cc:107] Waiting for socket to connect
0812 06:59:17.805280 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 06:59:17.805438 (+ 158us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:59:17.805707 (+ 269us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:59:17.805709 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:59:17.805726 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 06:59:17.805857 (+ 131us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:59:17.805862 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:59:17.806616 (+ 754us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:59:17.806621 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:59:17.807465 (+ 844us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:59:17.807474 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:59:17.808152 (+ 678us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:59:17.808156 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:59:17.808297 (+ 141us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:59:17.808316 (+ 19us) client_negotiation.cc:769] Sending connection context
0812 06:59:17.808405 (+ 89us) client_negotiation.cc:240] Negotiation successful
0812 06:59:17.808477 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":316,"mutex_wait_us":3,"spinlock_wait_cycles":14592,"thread_start_us":114,"threads_started":1}
I20260812 06:59:17.808704 10380 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:59:17.804944 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.52.65:44493 (local address 127.19.52.67:42151)
0812 06:59:17.805069 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0812 06:59:17.805073 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 06:59:17.805352 (+ 279us) server_negotiation.cc:407] Connection header received
0812 06:59:17.805595 (+ 243us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:59:17.805602 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:59:17.805646 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:59:17.805729 (+ 83us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 06:59:17.806024 (+ 295us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:59:17.806528 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:59:17.807629 (+ 1101us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:59:17.808047 (+ 418us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:59:17.808180 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:59:17.808228 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0812 06:59:17.808479 (+ 251us) server_negotiation.cc:299] Negotiation successful
0812 06:59:17.808584 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260812 06:59:17.808954 10392 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:59:17.804876 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.67:42151 (local address 127.19.52.65:44493)
0812 06:59:17.805231 (+ 355us) negotiation.cc:107] Waiting for socket to connect
0812 06:59:17.805250 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0812 06:59:17.805469 (+ 219us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:59:17.805751 (+ 282us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:59:17.805754 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:59:17.805770 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 06:59:17.805907 (+ 137us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:59:17.805912 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:59:17.806640 (+ 728us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:59:17.806645 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:59:17.807497 (+ 852us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:59:17.807506 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:59:17.808176 (+ 670us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:59:17.808180 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:59:17.808339 (+ 159us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:59:17.808356 (+ 17us) client_negotiation.cc:769] Sending connection context
0812 06:59:17.808440 (+ 84us) client_negotiation.cc:240] Negotiation successful
0812 06:59:17.808515 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":225,"mutex_wait_us":49,"spinlock_wait_cycles":95360,"thread_start_us":81,"threads_started":1}
I20260812 06:59:17.809587 10154 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3c2f304ec35e4bdbb209f5bf1c1c2b62" candidate_uuid: "03563d6c67c147a6896392f09207b598" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "07bc89147f6743668cd971afde67c3b9" is_pre_election: true
I20260812 06:59:17.809671 10310 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3c2f304ec35e4bdbb209f5bf1c1c2b62" candidate_uuid: "03563d6c67c147a6896392f09207b598" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dd60837110e8419f8feac79220e0508c" is_pre_election: true
I20260812 06:59:17.809933 10154 raft_consensus.cc:2454] T 3c2f304ec35e4bdbb209f5bf1c1c2b62 P 07bc89147f6743668cd971afde67c3b9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 03563d6c67c147a6896392f09207b598 in term 0.
I20260812 06:59:17.809979 10310 raft_consensus.cc:2454] T 3c2f304ec35e4bdbb209f5bf1c1c2b62 P dd60837110e8419f8feac79220e0508c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 03563d6c67c147a6896392f09207b598 in term 0.
I20260812 06:59:17.810451 9923 leader_election.cc:302] T 3c2f304ec35e4bdbb209f5bf1c1c2b62 P 03563d6c67c147a6896392f09207b598 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 03563d6c67c147a6896392f09207b598, 07bc89147f6743668cd971afde67c3b9; no voters:
I20260812 06:59:17.810730 10390 raft_consensus.cc:2781] T 3c2f304ec35e4bdbb209f5bf1c1c2b62 P 03563d6c67c147a6896392f09207b598 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 06:59:17.810820 10390 raft_consensus.cc:482] T 3c2f304ec35e4bdbb209f5bf1c1c2b62 P 03563d6c67c147a6896392f09207b598 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 06:59:17.810863 10390 raft_consensus.cc:3037] T 3c2f304ec35e4bdbb209f5bf1c1c2b62 P 03563d6c67c147a6896392f09207b598 [term 0 FOLLOWER]: Advancing to term 1
I20260812 06:59:17.811960 10390 raft_consensus.cc:504] T 3c2f304ec35e4bdbb209f5bf1c1c2b62 P 03563d6c67c147a6896392f09207b598 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03563d6c67c147a6896392f09207b598" member_type: VOTER last_known_addr { host: "127.19.52.65" port: 35197 } } peers { permanent_uuid: "07bc89147f6743668cd971afde67c3b9" member_type: VOTER last_known_addr { host: "127.19.52.66" port: 36125 } } peers { permanent_uuid: "dd60837110e8419f8feac79220e0508c" member_type: VOTER last_known_addr { host: "127.19.52.67" port: 42151 } }
I20260812 06:59:17.812577 10390 leader_election.cc:288] T 3c2f304ec35e4bdbb209f5bf1c1c2b62 P 03563d6c67c147a6896392f09207b598 [CANDIDATE]: Term 1 election: Requested vote from peers 07bc89147f6743668cd971afde67c3b9 (127.19.52.66:36125), dd60837110e8419f8feac79220e0508c (127.19.52.67:42151)
I20260812 06:59:17.812932 10154 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3c2f304ec35e4bdbb209f5bf1c1c2b62" candidate_uuid: "03563d6c67c147a6896392f09207b598" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "07bc89147f6743668cd971afde67c3b9"
I20260812 06:59:17.813074 10154 raft_consensus.cc:3037] T 3c2f304ec35e4bdbb209f5bf1c1c2b62 P 07bc89147f6743668cd971afde67c3b9 [term 0 FOLLOWER]: Advancing to term 1
I20260812 06:59:17.813154 10310 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3c2f304ec35e4bdbb209f5bf1c1c2b62" candidate_uuid: "03563d6c67c147a6896392f09207b598" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dd60837110e8419f8feac79220e0508c"
I20260812 06:59:17.813313 10310 raft_consensus.cc:3037] T 3c2f304ec35e4bdbb209f5bf1c1c2b62 P dd60837110e8419f8feac79220e0508c [term 0 FOLLOWER]: Advancing to term 1
I20260812 06:59:17.814244 10154 raft_consensus.cc:2454] T 3c2f304ec35e4bdbb209f5bf1c1c2b62 P 07bc89147f6743668cd971afde67c3b9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 03563d6c67c147a6896392f09207b598 in term 1.
I20260812 06:59:17.814380 10310 raft_consensus.cc:2454] T 3c2f304ec35e4bdbb209f5bf1c1c2b62 P dd60837110e8419f8feac79220e0508c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 03563d6c67c147a6896392f09207b598 in term 1.
I20260812 06:59:17.814626 9923 leader_election.cc:302] T 3c2f304ec35e4bdbb209f5bf1c1c2b62 P 03563d6c67c147a6896392f09207b598 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 03563d6c67c147a6896392f09207b598, 07bc89147f6743668cd971afde67c3b9; no voters:
I20260812 06:59:17.814796 10390 raft_consensus.cc:2781] T 3c2f304ec35e4bdbb209f5bf1c1c2b62 P 03563d6c67c147a6896392f09207b598 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 06:59:17.815073 10390 raft_consensus.cc:686] T 3c2f304ec35e4bdbb209f5bf1c1c2b62 P 03563d6c67c147a6896392f09207b598 [term 1 LEADER]: Becoming Leader. State: Replica: 03563d6c67c147a6896392f09207b598, State: Running, Role: LEADER
I20260812 06:59:17.815423 10390 consensus_queue.cc:237] T 3c2f304ec35e4bdbb209f5bf1c1c2b62 P 03563d6c67c147a6896392f09207b598 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03563d6c67c147a6896392f09207b598" member_type: VOTER last_known_addr { host: "127.19.52.65" port: 35197 } } peers { permanent_uuid: "07bc89147f6743668cd971afde67c3b9" member_type: VOTER last_known_addr { host: "127.19.52.66" port: 36125 } } peers { permanent_uuid: "dd60837110e8419f8feac79220e0508c" member_type: VOTER last_known_addr { host: "127.19.52.67" port: 42151 } }
I20260812 06:59:17.819032 9836 catalog_manager.cc:5495] T 3c2f304ec35e4bdbb209f5bf1c1c2b62 P 03563d6c67c147a6896392f09207b598 reported cstate change: term changed from 0 to 1, leader changed from <none> to 03563d6c67c147a6896392f09207b598 (127.19.52.65). New cstate: current_term: 1 leader_uuid: "03563d6c67c147a6896392f09207b598" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03563d6c67c147a6896392f09207b598" member_type: VOTER last_known_addr { host: "127.19.52.65" port: 35197 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "07bc89147f6743668cd971afde67c3b9" member_type: VOTER last_known_addr { host: "127.19.52.66" port: 36125 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dd60837110e8419f8feac79220e0508c" member_type: VOTER last_known_addr { host: "127.19.52.67" port: 42151 } health_report { overall_health: UNKNOWN } } }
W20260812 06:59:17.819720 10212 tablet.cc:2369] T 3c2f304ec35e4bdbb209f5bf1c1c2b62 P 07bc89147f6743668cd971afde67c3b9: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 06:59:17.866866 10377 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:59:17.863045 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57592 (local address 127.19.52.65:35197)
0812 06:59:17.863192 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0812 06:59:17.863198 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 06:59:17.863216 (+ 18us) server_negotiation.cc:407] Connection header received
0812 06:59:17.863422 (+ 206us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:59:17.863425 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:59:17.863486 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:59:17.863613 (+ 127us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 06:59:17.863871 (+ 258us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:59:17.864389 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:59:17.865328 (+ 939us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:59:17.865566 (+ 238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:59:17.865683 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:59:17.866012 (+ 329us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 06:59:17.866436 (+ 424us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 06:59:17.866580 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0812 06:59:17.866651 (+ 71us) server_negotiation.cc:299] Negotiation successful
0812 06:59:17.866712 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260812 06:59:17.873210 10154 raft_consensus.cc:1261] T 3c2f304ec35e4bdbb209f5bf1c1c2b62 P 07bc89147f6743668cd971afde67c3b9 [term 1 FOLLOWER]: Refusing update from remote peer 03563d6c67c147a6896392f09207b598: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 06:59:17.873219 10310 raft_consensus.cc:1261] T 3c2f304ec35e4bdbb209f5bf1c1c2b62 P dd60837110e8419f8feac79220e0508c [term 1 FOLLOWER]: Refusing update from remote peer 03563d6c67c147a6896392f09207b598: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 06:59:17.874011 10394 consensus_queue.cc:1035] T 3c2f304ec35e4bdbb209f5bf1c1c2b62 P 03563d6c67c147a6896392f09207b598 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd60837110e8419f8feac79220e0508c" member_type: VOTER last_known_addr { host: "127.19.52.67" port: 42151 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 06:59:17.874234 10390 consensus_queue.cc:1035] T 3c2f304ec35e4bdbb209f5bf1c1c2b62 P 03563d6c67c147a6896392f09207b598 [LEADER]: Connected to new peer: Peer: permanent_uuid: "07bc89147f6743668cd971afde67c3b9" member_type: VOTER last_known_addr { host: "127.19.52.66" port: 36125 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 06:59:17.883068 10400 mvcc.cc:205] Tried to move back new op lower bound from 7317577555440750592 to 7317577555214868480. Current Snapshot: MvccSnapshot[applied={T|T < 7317577555440750592}]
I20260812 06:59:17.883147 10398 mvcc.cc:205] Tried to move back new op lower bound from 7317577555440750592 to 7317577555214868480. Current Snapshot: MvccSnapshot[applied={T|T < 7317577555440750592}]
I20260812 06:59:17.883291 10399 mvcc.cc:205] Tried to move back new op lower bound from 7317577555440750592 to 7317577555214868480. Current Snapshot: MvccSnapshot[applied={T|T < 7317577555440750592}]
I20260812 06:59:17.892097 9837 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:34680:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 06:59:17.892313 9837 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:34680:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 06:59:17.895082 9837 catalog_manager.cc:5782] T 00000000000000000000000000000000 P c3fd97b36e534f3496d74e809a8a6056: Sending DeleteTablet for 3 replicas of tablet 3c2f304ec35e4bdbb209f5bf1c1c2b62
I20260812 06:59:17.896080 10289 tablet_service.cc:1514] Processing DeleteTablet for tablet 3c2f304ec35e4bdbb209f5bf1c1c2b62 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 06:59:17 UTC) from {username='slave'} at 127.0.0.1:53390
I20260812 06:59:17.896047 10132 tablet_service.cc:1514] Processing DeleteTablet for tablet 3c2f304ec35e4bdbb209f5bf1c1c2b62 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 06:59:17 UTC) from {username='slave'} at 127.0.0.1:49844
I20260812 06:59:17.896708 9979 tablet_service.cc:1514] Processing DeleteTablet for tablet 3c2f304ec35e4bdbb209f5bf1c1c2b62 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 06:59:17 UTC) from {username='slave'} at 127.0.0.1:57586
I20260812 06:59:17.896951 10410 tablet_replica.cc:330] T 3c2f304ec35e4bdbb209f5bf1c1c2b62 P dd60837110e8419f8feac79220e0508c: stopping tablet replica
I20260812 06:59:17.897125 10409 tablet_replica.cc:330] T 3c2f304ec35e4bdbb209f5bf1c1c2b62 P 07bc89147f6743668cd971afde67c3b9: stopping tablet replica
I20260812 06:59:17.897245 10411 tablet_replica.cc:330] T 3c2f304ec35e4bdbb209f5bf1c1c2b62 P 03563d6c67c147a6896392f09207b598: stopping tablet replica
I20260812 06:59:17.897534 19665 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu with pid 9906
I20260812 06:59:17.902077 10409 raft_consensus.cc:2229] T 3c2f304ec35e4bdbb209f5bf1c1c2b62 P 07bc89147f6743668cd971afde67c3b9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 06:59:17.902397 10409 raft_consensus.cc:2258] T 3c2f304ec35e4bdbb209f5bf1c1c2b62 P 07bc89147f6743668cd971afde67c3b9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 06:59:17.902853 10410 raft_consensus.cc:2229] T 3c2f304ec35e4bdbb209f5bf1c1c2b62 P dd60837110e8419f8feac79220e0508c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 06:59:17.903188 10410 raft_consensus.cc:2258] T 3c2f304ec35e4bdbb209f5bf1c1c2b62 P dd60837110e8419f8feac79220e0508c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 06:59:17.904307 10409 ts_tablet_manager.cc:1905] T 3c2f304ec35e4bdbb209f5bf1c1c2b62 P 07bc89147f6743668cd971afde67c3b9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 06:59:17.908118 10409 ts_tablet_manager.cc:1918] T 3c2f304ec35e4bdbb209f5bf1c1c2b62 P 07bc89147f6743668cd971afde67c3b9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 06:59:17.908248 10409 log.cc:1196] T 3c2f304ec35e4bdbb209f5bf1c1c2b62 P 07bc89147f6743668cd971afde67c3b9: Deleting WAL directory at /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786517687384214-19665-0/minicluster-data/ts-1/wal/wals/3c2f304ec35e4bdbb209f5bf1c1c2b62
I20260812 06:59:17.908634 10409 ts_tablet_manager.cc:1939] T 3c2f304ec35e4bdbb209f5bf1c1c2b62 P 07bc89147f6743668cd971afde67c3b9: Deleting consensus metadata
I20260812 06:59:17.909771 10410 ts_tablet_manager.cc:1905] T 3c2f304ec35e4bdbb209f5bf1c1c2b62 P dd60837110e8419f8feac79220e0508c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 06:59:17.910292 9819 catalog_manager.cc:4841] TS 07bc89147f6743668cd971afde67c3b9 (127.19.52.66:36125): tablet 3c2f304ec35e4bdbb209f5bf1c1c2b62 (table test-table [id=6e17801c7faa4226bc3242471f123963]) successfully deleted
W20260812 06:59:17.910851 9819 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.19.52.65:35197 (error 108)
I20260812 06:59:17.911103 19665 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu with pid 10058
I20260812 06:59:17.913390 10410 ts_tablet_manager.cc:1918] T 3c2f304ec35e4bdbb209f5bf1c1c2b62 P dd60837110e8419f8feac79220e0508c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 06:59:17.913556 10410 log.cc:1196] T 3c2f304ec35e4bdbb209f5bf1c1c2b62 P dd60837110e8419f8feac79220e0508c: Deleting WAL directory at /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786517687384214-19665-0/minicluster-data/ts-2/wal/wals/3c2f304ec35e4bdbb209f5bf1c1c2b62
I20260812 06:59:17.913959 10410 ts_tablet_manager.cc:1939] T 3c2f304ec35e4bdbb209f5bf1c1c2b62 P dd60837110e8419f8feac79220e0508c: Deleting consensus metadata
I20260812 06:59:17.914330 10376 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:59:17.913914 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.65:35197 (local address 127.0.0.1:57594)
0812 06:59:17.914135 (+ 221us) negotiation.cc:107] Waiting for socket to connect
0812 06:59:17.914220 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.52.65:35197: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":115}
W20260812 06:59:17.914551 9819 catalog_manager.cc:4584] TS 03563d6c67c147a6896392f09207b598 (127.19.52.65:35197): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 3c2f304ec35e4bdbb209f5bf1c1c2b62: Network error: Client connection negotiation failed: client connection to 127.19.52.65:35197: connect: Connection refused (error 111)
I20260812 06:59:17.915470 9820 catalog_manager.cc:4841] TS dd60837110e8419f8feac79220e0508c (127.19.52.67:42151): tablet 3c2f304ec35e4bdbb209f5bf1c1c2b62 (table test-table [id=6e17801c7faa4226bc3242471f123963]) successfully deleted
I20260812 06:59:17.919826 19665 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu with pid 10214
I20260812 06:59:17.928323 19665 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu with pid 9800
2026-08-12T06:59:17Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/5 (1295 ms)
[----------] 6 tests from EncryptionPolicyTest (11664 ms total)
[----------] 18 tests from AuthTokenIssuingTest
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/0
Loading random data
Initializing database '/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786517687384214-19665-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 06:59:18 dist-test-slave-65mx krb5kdc[10415](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 06:59:18 dist-test-slave-65mx krb5kdc[10415](info): set up 2 sockets
Aug 12 06:59:18 dist-test-slave-65mx krb5kdc[10415](info): commencing operation
krb5kdc: starting...
W20260812 06:59:19.069083 19665 mini_kdc.cc:121] Time spent starting KDC: real 1.115s user 0.002s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786517687384214-19665-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786517687384214-19665-0/krb5kdc/test-user.keytab.
Aug 12 06:59:19 dist-test-slave-65mx krb5kdc[10415](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517959, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T06:59:19Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T06:59:19Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786517687384214-19665-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786517687384214-19665-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260812 06:59:19.216013 19665 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu
/tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786517687384214-19665-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786517687384214-19665-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786517687384214-19665-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786517687384214-19665-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36733
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786517687384214-19665-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:44229
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:36733
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786517687384214-19665-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786517687384214-19665-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786517687384214-19665-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 06:59:19.333786 10430 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:59:19.334255 10430 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:59:19.334353 10430 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 06:59:19.339051 10430 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 06:59:19.339180 10430 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 06:59:19.339219 10430 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 06:59:19.339262 10430 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 06:59:19.339303 10430 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 06:59:19.345640 10430 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:44229
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786517687384214-19665-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786517687384214-19665-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:36733
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786517687384214-19665-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36733
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786517687384214-19665-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10430
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786517687384214-19665-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 06:15:54 UTC on ccycloud.quasar-rnfzce.root.comops.site
build id 4581989
I20260812 06:59:19.347008 10430 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:59:19.348228 10430 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 06:59:19.357873 10437 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:59:19.357817 10435 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:59:19.357868 10439 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:59:19.358018 10430 server_base.cc:1034] running on GCE node
I20260812 06:59:19.358887 10430 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:59:19.360126 10430 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:59:19.361308 10430 hybrid_clock.cc:648] HybridClock initialized: now 1786517959361263 us; error 68 us; skew 500 ppm
Aug 12 06:59:19 dist-test-slave-65mx krb5kdc[10415](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517959, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 06:59:19.364427 10430 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260812 06:59:19.365442 10430 webserver.cc:460] Webserver started at http://127.0.0.1:36083/ using document root <none> and password file <none>
I20260812 06:59:19.365998 10430 fs_manager.cc:359] Metadata directory not provided
I20260812 06:59:19.366063 10430 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:59:19.366235 10430 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 06:59:19.368362 10430 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786517687384214-19665-0/minicluster-data/master-0/data/instance:
uuid: "e308fac8c87c435a8ef676145b935338"
format_stamp: "Formatted at 2026-08-12 06:59:19 on dist-test-slave-65mx"
server_key: "abbb4eb4e323fe06c41b70cd7a4525de"
server_key_iv: "327e3534486e53c24f79a7d10c54facc"
server_key_version: "encryptionkey@0"
I20260812 06:59:19.368991 10430 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786517687384214-19665-0/minicluster-data/master-0/wal/instance:
uuid: "e308fac8c87c435a8ef676145b935338"
format_stamp: "Formatted at 2026-08-12 06:59:19 on dist-test-slave-65mx"
server_key: "abbb4eb4e323fe06c41b70cd7a4525de"
server_key_iv: "327e3534486e53c24f79a7d10c54facc"
server_key_version: "encryptionkey@0"
I20260812 06:59:19.372952 10430 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260812 06:59:19.375686 10448 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 06:59:19.377352 10430 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 06:59:19.377533 10430 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786517687384214-19665-0/minicluster-data/master-0/data,/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786517687384214-19665-0/minicluster-data/master-0/wal
uuid: "e308fac8c87c435a8ef676145b935338"
format_stamp: "Formatted at 2026-08-12 06:59:19 on dist-test-slave-65mx"
server_key: "abbb4eb4e323fe06c41b70cd7a4525de"
server_key_iv: "327e3534486e53c24f79a7d10c54facc"
server_key_version: "encryptionkey@0"
I20260812 06:59:19.377676 10430 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786517687384214-19665-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786517687384214-19665-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786517687384214-19665-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:59:19.393019 10430 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 06:59:19.397017 10430 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 06:59:19.397264 10430 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:59:19.405122 10430 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:36733
I20260812 06:59:19.405149 10508 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:36733 every 8 connection(s)
I20260812 06:59:19.406147 10430 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786517687384214-19665-0/minicluster-data/master-0/data/info.pb
I20260812 06:59:19.410386 10509 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 06:59:19.414767 19665 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu as pid 10430
I20260812 06:59:19.414927 19665 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786517687384214-19665-0/minicluster-data/master-0/wal/instance
I20260812 06:59:19.415295 19665 external_mini_cluster.cc:1428] Setting key 8191649ec909d42cee315ae7506f0ff4
I20260812 06:59:19.417900 10509 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e308fac8c87c435a8ef676145b935338: Bootstrap starting.
I20260812 06:59:19.420828 10509 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e308fac8c87c435a8ef676145b935338: Neither blocks nor log segments found. Creating new log.
I20260812 06:59:19.421698 10509 log.cc:824] T 00000000000000000000000000000000 P e308fac8c87c435a8ef676145b935338: Log is configured to *not* fsync() on all Append() calls
I20260812 06:59:19.424113 10509 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e308fac8c87c435a8ef676145b935338: No bootstrap required, opened a new log
Aug 12 06:59:19 dist-test-slave-65mx krb5kdc[10415](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517959, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260812 06:59:19.428097 10509 raft_consensus.cc:348] T 00000000000000000000000000000000 P e308fac8c87c435a8ef676145b935338 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e308fac8c87c435a8ef676145b935338" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36733 } }
I20260812 06:59:19.428402 10509 raft_consensus.cc:374] T 00000000000000000000000000000000 P e308fac8c87c435a8ef676145b935338 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 06:59:19.428454 10509 raft_consensus.cc:729] T 00000000000000000000000000000000 P e308fac8c87c435a8ef676145b935338 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e308fac8c87c435a8ef676145b935338, State: Initialized, Role: FOLLOWER
I20260812 06:59:19.428988 10509 consensus_queue.cc:260] T 00000000000000000000000000000000 P e308fac8c87c435a8ef676145b935338 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e308fac8c87c435a8ef676145b935338" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36733 } }
I20260812 06:59:19.429176 10509 raft_consensus.cc:388] T 00000000000000000000000000000000 P e308fac8c87c435a8ef676145b935338 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 06:59:19.429255 10509 raft_consensus.cc:482] T 00000000000000000000000000000000 P e308fac8c87c435a8ef676145b935338 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 06:59:19.429354 10509 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e308fac8c87c435a8ef676145b935338 [term 0 FOLLOWER]: Advancing to term 1
I20260812 06:59:19.430629 10509 raft_consensus.cc:504] T 00000000000000000000000000000000 P e308fac8c87c435a8ef676145b935338 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e308fac8c87c435a8ef676145b935338" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36733 } }
I20260812 06:59:19.431031 10509 leader_election.cc:302] T 00000000000000000000000000000000 P e308fac8c87c435a8ef676145b935338 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e308fac8c87c435a8ef676145b935338; no voters:
I20260812 06:59:19.431433 10509 leader_election.cc:288] T 00000000000000000000000000000000 P e308fac8c87c435a8ef676145b935338 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 06:59:19.431651 10515 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e308fac8c87c435a8ef676145b935338 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 06:59:19.431977 10515 raft_consensus.cc:686] T 00000000000000000000000000000000 P e308fac8c87c435a8ef676145b935338 [term 1 LEADER]: Becoming Leader. State: Replica: e308fac8c87c435a8ef676145b935338, State: Running, Role: LEADER
I20260812 06:59:19.432397 10515 consensus_queue.cc:237] T 00000000000000000000000000000000 P e308fac8c87c435a8ef676145b935338 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e308fac8c87c435a8ef676145b935338" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36733 } }
I20260812 06:59:19.432722 10512 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:59:19.416777 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34427 (local address 127.0.0.1:36733)
0812 06:59:19.417261 (+ 484us) server_negotiation.cc:206] Beginning negotiation
0812 06:59:19.417281 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0812 06:59:19.417322 (+ 41us) server_negotiation.cc:407] Connection header received
0812 06:59:19.418198 (+ 876us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:59:19.418221 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:59:19.418734 (+ 513us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:59:19.419105 (+ 371us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:59:19.419818 (+ 713us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:59:19.421317 (+ 1499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:59:19.422605 (+ 1288us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:59:19.423270 (+ 665us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:59:19.423458 (+ 188us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:59:19.426173 (+ 2715us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:59:19.426182 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:59:19.426201 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:59:19.426235 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:59:19.429016 (+ 2781us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:59:19.429721 (+ 705us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:59:19.429727 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:59:19.429733 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:59:19.429868 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:59:19.430196 (+ 328us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:59:19.430200 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:59:19.430201 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:59:19.430802 (+ 601us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:59:19.431003 (+ 201us) server_negotiation.cc:1035] Waiting for connection context
0812 06:59:19.431402 (+ 399us) server_negotiation.cc:299] Negotiation successful
0812 06:59:19.431658 (+ 256us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":335,"thread_start_us":161,"threads_started":1}
I20260812 06:59:19.432852 10509 sys_catalog.cc:564] T 00000000000000000000000000000000 P e308fac8c87c435a8ef676145b935338 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 06:59:19.434064 10515 sys_catalog.cc:455] T 00000000000000000000000000000000 P e308fac8c87c435a8ef676145b935338 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e308fac8c87c435a8ef676145b935338. Latest consensus state: current_term: 1 leader_uuid: "e308fac8c87c435a8ef676145b935338" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e308fac8c87c435a8ef676145b935338" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36733 } } }
I20260812 06:59:19.434222 10515 sys_catalog.cc:458] T 00000000000000000000000000000000 P e308fac8c87c435a8ef676145b935338 [sys.catalog]: This master's current role is: LEADER
I20260812 06:59:19.434561 10517 sys_catalog.cc:455] T 00000000000000000000000000000000 P e308fac8c87c435a8ef676145b935338 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e308fac8c87c435a8ef676145b935338" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e308fac8c87c435a8ef676145b935338" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36733 } } }
I20260812 06:59:19.434681 10517 sys_catalog.cc:458] T 00000000000000000000000000000000 P e308fac8c87c435a8ef676145b935338 [sys.catalog]: This master's current role is: LEADER
I20260812 06:59:19.436520 19665 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
W20260812 06:59:19.438460 10534 catalog_manager.cc:1558] T 00000000000000000000000000000000 P e308fac8c87c435a8ef676145b935338: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 06:59:19.438577 10534 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 06:59:19.438681 10527 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 06:59:19.439460 10527 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 06:59:19.446180 10527 catalog_manager.cc:1347] Generated new cluster ID: 2f0dc20577914020b2e93fb830d4fcf0
I20260812 06:59:19.446261 10527 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 06:59:19.453729 10527 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 06:59:19.455055 10527 catalog_manager.cc:1504] Loading token signing keys...
I20260812 06:59:19.457029 10512 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:59:19.440960 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35475 (local address 127.0.0.1:36733)
0812 06:59:19.441081 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0812 06:59:19.441088 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 06:59:19.441897 (+ 809us) server_negotiation.cc:407] Connection header received
0812 06:59:19.442014 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:59:19.442018 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:59:19.442072 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:59:19.442170 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:59:19.443064 (+ 894us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:59:19.443902 (+ 838us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:59:19.445089 (+ 1187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:59:19.445472 (+ 383us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:59:19.445600 (+ 128us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:59:19.452866 (+ 7266us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:59:19.452870 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:59:19.452873 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:59:19.452911 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:59:19.454637 (+ 1726us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:59:19.455265 (+ 628us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:59:19.455269 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:59:19.455271 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:59:19.455360 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:59:19.455902 (+ 542us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:59:19.455906 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:59:19.455908 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:59:19.456129 (+ 221us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:59:19.456314 (+ 185us) server_negotiation.cc:1035] Waiting for connection context
0812 06:59:19.456695 (+ 381us) server_negotiation.cc:299] Negotiation successful
0812 06:59:19.456804 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260812 06:59:19.460443 10527 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e308fac8c87c435a8ef676145b935338: Generated new TSK 0
I20260812 06:59:19.461333 10527 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 06:59:19.467067 19665 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu with pid 10430
W20260812 06:59:19.475647 10413 connection.cc:535] client connection to 127.0.0.1:36733 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:36733): Connection reset by peer (error 104)
2026-08-12T06:59:19Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/0 (1536 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/1
Loading random data
Initializing database '/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786517687384214-19665-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 06:59:19 dist-test-slave-65mx krb5kdc[10547](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 06:59:19 dist-test-slave-65mx krb5kdc[10547](info): set up 2 sockets
Aug 12 06:59:19 dist-test-slave-65mx krb5kdc[10547](info): commencing operation
krb5kdc: starting...
W20260812 06:59:20.598484 19665 mini_kdc.cc:121] Time spent starting KDC: real 1.108s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786517687384214-19665-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786517687384214-19665-0/krb5kdc/test-user.keytab.
Aug 12 06:59:20 dist-test-slave-65mx krb5kdc[10547](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517960, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T06:59:20Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T06:59:20Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786517687384214-19665-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786517687384214-19665-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260812 06:59:20.740953 19665 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu
/tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786517687384214-19665-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786517687384214-19665-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786517687384214-19665-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786517687384214-19665-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44593
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786517687384214-19665-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:39469
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:44593
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786517687384214-19665-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786517687384214-19665-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786517687384214-19665-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 06:59:20.857307 10561 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:59:20.857636 10561 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:59:20.857719 10561 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 06:59:20.861855 10561 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 06:59:20.861958 10561 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 06:59:20.861994 10561 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 06:59:20.862033 10561 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 06:59:20.862072 10561 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 06:59:20.867507 10561 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:39469
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786517687384214-19665-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786517687384214-19665-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:44593
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786517687384214-19665-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44593
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786517687384214-19665-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10561
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786517687384214-19665-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 06:15:54 UTC on ccycloud.quasar-rnfzce.root.comops.site
build id 4581989
I20260812 06:59:20.868667 10561 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:59:20.869736 10561 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 06:59:20.878350 10570 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:59:20.878352 10567 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:59:20.878592 10561 server_base.cc:1034] running on GCE node
W20260812 06:59:20.878336 10568 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:59:20.879284 10561 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:59:20.880437 10561 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:59:20.881647 10561 hybrid_clock.cc:648] HybridClock initialized: now 1786517960881617 us; error 41 us; skew 500 ppm
Aug 12 06:59:20 dist-test-slave-65mx krb5kdc[10547](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517960, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 06:59:20.884536 10561 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260812 06:59:20.885569 10561 webserver.cc:460] Webserver started at http://127.0.0.1:44545/ using document root <none> and password file <none>
I20260812 06:59:20.886101 10561 fs_manager.cc:359] Metadata directory not provided
I20260812 06:59:20.886159 10561 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:59:20.886327 10561 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 06:59:20.888375 10561 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786517687384214-19665-0/minicluster-data/master-0/data/instance:
uuid: "741b63c4ea504458a3f4f90d244afb22"
format_stamp: "Formatted at 2026-08-12 06:59:20 on dist-test-slave-65mx"
server_key: "eb43866707b8708981dfd46ef35d34ee"
server_key_iv: "de9b69a47783cee07fcc5e2cd517a8c9"
server_key_version: "encryptionkey@0"
I20260812 06:59:20.889009 10561 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786517687384214-19665-0/minicluster-data/master-0/wal/instance:
uuid: "741b63c4ea504458a3f4f90d244afb22"
format_stamp: "Formatted at 2026-08-12 06:59:20 on dist-test-slave-65mx"
server_key: "eb43866707b8708981dfd46ef35d34ee"
server_key_iv: "de9b69a47783cee07fcc5e2cd517a8c9"
server_key_version: "encryptionkey@0"
I20260812 06:59:20.892897 10561 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260812 06:59:20.895375 10579 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 06:59:20.896905 10561 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 06:59:20.897064 10561 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786517687384214-19665-0/minicluster-data/master-0/data,/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786517687384214-19665-0/minicluster-data/master-0/wal
uuid: "741b63c4ea504458a3f4f90d244afb22"
format_stamp: "Formatted at 2026-08-12 06:59:20 on dist-test-slave-65mx"
server_key: "eb43866707b8708981dfd46ef35d34ee"
server_key_iv: "de9b69a47783cee07fcc5e2cd517a8c9"
server_key_version: "encryptionkey@0"
I20260812 06:59:20.897181 10561 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786517687384214-19665-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786517687384214-19665-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786517687384214-19665-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:59:20.931423 10561 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 06:59:20.935153 10561 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 06:59:20.935405 10561 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:59:20.943540 10561 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:44593
I20260812 06:59:20.943573 10640 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:44593 every 8 connection(s)
I20260812 06:59:20.944600 10561 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786517687384214-19665-0/minicluster-data/master-0/data/info.pb
I20260812 06:59:20.947562 10641 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 06:59:20.949435 19665 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu as pid 10561
I20260812 06:59:20.949590 19665 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786517687384214-19665-0/minicluster-data/master-0/wal/instance
I20260812 06:59:20.949916 19665 external_mini_cluster.cc:1428] Setting key c169ac4d2d925aa3abf5fe44d9771ec4
I20260812 06:59:20.954454 10641 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 741b63c4ea504458a3f4f90d244afb22: Bootstrap starting.
I20260812 06:59:20.957362 10641 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 741b63c4ea504458a3f4f90d244afb22: Neither blocks nor log segments found. Creating new log.
I20260812 06:59:20.958230 10641 log.cc:824] T 00000000000000000000000000000000 P 741b63c4ea504458a3f4f90d244afb22: Log is configured to *not* fsync() on all Append() calls
I20260812 06:59:20.959563 10644 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:59:20.951356 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56739 (local address 127.0.0.1:44593)
0812 06:59:20.951926 (+ 570us) server_negotiation.cc:206] Beginning negotiation
0812 06:59:20.951942 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0812 06:59:20.951979 (+ 37us) server_negotiation.cc:407] Connection header received
0812 06:59:20.952679 (+ 700us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:59:20.952697 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:59:20.953102 (+ 405us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:59:20.953386 (+ 284us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:59:20.953786 (+ 400us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:59:20.954853 (+ 1067us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:59:20.956053 (+ 1200us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:59:20.956509 (+ 456us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:59:20.956729 (+ 220us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:59:20.957759 (+ 1030us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:59:20.957764 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:59:20.957778 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 06:59:20.958185 (+ 407us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:59:20.958245 (+ 60us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 06:59:20.958268 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:59:20.958439 (+ 171us) server_negotiation.cc:1035] Waiting for connection context
0812 06:59:20.958654 (+ 215us) server_negotiation.cc:299] Negotiation successful
0812 06:59:20.958805 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":417,"thread_start_us":175,"threads_started":1}
I20260812 06:59:20.960309 10641 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 741b63c4ea504458a3f4f90d244afb22: No bootstrap required, opened a new log
I20260812 06:59:20.962786 19665 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 06:59:20.964821 10641 raft_consensus.cc:348] T 00000000000000000000000000000000 P 741b63c4ea504458a3f4f90d244afb22 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "741b63c4ea504458a3f4f90d244afb22" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44593 } }
I20260812 06:59:20.965148 10641 raft_consensus.cc:374] T 00000000000000000000000000000000 P 741b63c4ea504458a3f4f90d244afb22 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 06:59:20.965236 10641 raft_consensus.cc:729] T 00000000000000000000000000000000 P 741b63c4ea504458a3f4f90d244afb22 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 741b63c4ea504458a3f4f90d244afb22, State: Initialized, Role: FOLLOWER
I20260812 06:59:20.965746 10641 consensus_queue.cc:260] T 00000000000000000000000000000000 P 741b63c4ea504458a3f4f90d244afb22 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "741b63c4ea504458a3f4f90d244afb22" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44593 } }
I20260812 06:59:20.965948 10641 raft_consensus.cc:388] T 00000000000000000000000000000000 P 741b63c4ea504458a3f4f90d244afb22 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 06:59:20.966030 10641 raft_consensus.cc:482] T 00000000000000000000000000000000 P 741b63c4ea504458a3f4f90d244afb22 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 06:59:20.966132 10641 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 741b63c4ea504458a3f4f90d244afb22 [term 0 FOLLOWER]: Advancing to term 1
I20260812 06:59:20.967262 10641 raft_consensus.cc:504] T 00000000000000000000000000000000 P 741b63c4ea504458a3f4f90d244afb22 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "741b63c4ea504458a3f4f90d244afb22" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44593 } }
I20260812 06:59:20.967621 10641 leader_election.cc:302] T 00000000000000000000000000000000 P 741b63c4ea504458a3f4f90d244afb22 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 741b63c4ea504458a3f4f90d244afb22; no voters:
I20260812 06:59:20.967972 10641 leader_election.cc:288] T 00000000000000000000000000000000 P 741b63c4ea504458a3f4f90d244afb22 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 06:59:20.968055 10652 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 741b63c4ea504458a3f4f90d244afb22 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 06:59:20.968255 10652 raft_consensus.cc:686] T 00000000000000000000000000000000 P 741b63c4ea504458a3f4f90d244afb22 [term 1 LEADER]: Becoming Leader. State: Replica: 741b63c4ea504458a3f4f90d244afb22, State: Running, Role: LEADER
I20260812 06:59:20.968634 10652 consensus_queue.cc:237] T 00000000000000000000000000000000 P 741b63c4ea504458a3f4f90d244afb22 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "741b63c4ea504458a3f4f90d244afb22" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44593 } }
I20260812 06:59:20.969399 10641 sys_catalog.cc:564] T 00000000000000000000000000000000 P 741b63c4ea504458a3f4f90d244afb22 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 06:59:20.969448 10644 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:59:20.964934 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51987 (local address 127.0.0.1:44593)
0812 06:59:20.965050 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0812 06:59:20.965054 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 06:59:20.965361 (+ 307us) server_negotiation.cc:407] Connection header received
0812 06:59:20.965539 (+ 178us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:59:20.965543 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:59:20.965594 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:59:20.965677 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:59:20.965919 (+ 242us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:59:20.966674 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:59:20.967507 (+ 833us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:59:20.967807 (+ 300us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:59:20.967931 (+ 124us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:59:20.968948 (+ 1017us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:59:20.968951 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:59:20.968954 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 06:59:20.968967 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:59:20.969001 (+ 34us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 06:59:20.969013 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:59:20.969119 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0812 06:59:20.969254 (+ 135us) server_negotiation.cc:299] Negotiation successful
0812 06:59:20.969293 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260812 06:59:20.970613 10653 sys_catalog.cc:455] T 00000000000000000000000000000000 P 741b63c4ea504458a3f4f90d244afb22 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 741b63c4ea504458a3f4f90d244afb22. Latest consensus state: current_term: 1 leader_uuid: "741b63c4ea504458a3f4f90d244afb22" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "741b63c4ea504458a3f4f90d244afb22" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44593 } } }
I20260812 06:59:20.970615 10655 sys_catalog.cc:455] T 00000000000000000000000000000000 P 741b63c4ea504458a3f4f90d244afb22 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "741b63c4ea504458a3f4f90d244afb22" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "741b63c4ea504458a3f4f90d244afb22" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44593 } } }
I20260812 06:59:20.970772 10655 sys_catalog.cc:458] T 00000000000000000000000000000000 P 741b63c4ea504458a3f4f90d244afb22 [sys.catalog]: This master's current role is: LEADER
I20260812 06:59:20.970772 10653 sys_catalog.cc:458] T 00000000000000000000000000000000 P 741b63c4ea504458a3f4f90d244afb22 [sys.catalog]: This master's current role is: LEADER
I20260812 06:59:20.971242 10662 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 06:59:20.974992 10662 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 06:59:20.981035 10662 catalog_manager.cc:1347] Generated new cluster ID: 4228a3e992444fd0babeb59852a40b24
I20260812 06:59:20.981124 10662 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 06:59:21.004683 10662 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 06:59:21.005805 10662 catalog_manager.cc:1504] Loading token signing keys...
I20260812 06:59:21.013815 10662 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 741b63c4ea504458a3f4f90d244afb22: Generated new TSK 0
I20260812 06:59:21.014575 10662 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 06:59:21.020510 19665 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu with pid 10561
W20260812 06:59:21.029088 10545 connection.cc:535] client connection to 127.0.0.1:44593 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:44593): Connection reset by peer (error 104)
2026-08-12T06:59:21Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/1 (1553 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/2
Loading random data
Initializing database '/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786517687384214-19665-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 06:59:21 dist-test-slave-65mx krb5kdc[10675](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 06:59:21 dist-test-slave-65mx krb5kdc[10675](info): set up 2 sockets
Aug 12 06:59:21 dist-test-slave-65mx krb5kdc[10675](info): commencing operation
krb5kdc: starting...
W20260812 06:59:22.161134 19665 mini_kdc.cc:121] Time spent starting KDC: real 1.117s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786517687384214-19665-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786517687384214-19665-0/krb5kdc/test-user.keytab.
Aug 12 06:59:22 dist-test-slave-65mx krb5kdc[10675](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517962, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T06:59:22Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T06:59:22Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786517687384214-19665-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786517687384214-19665-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260812 06:59:22.303614 19665 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu
/tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786517687384214-19665-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786517687384214-19665-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786517687384214-19665-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786517687384214-19665-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43619
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786517687384214-19665-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:34435
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:43619
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786517687384214-19665-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786517687384214-19665-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786517687384214-19665-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 06:59:22.420687 10692 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:59:22.421031 10692 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:59:22.421131 10692 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 06:59:22.425405 10692 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 06:59:22.425496 10692 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 06:59:22.425535 10692 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 06:59:22.425572 10692 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 06:59:22.425606 10692 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 06:59:22.431227 10692 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:34435
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786517687384214-19665-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786517687384214-19665-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:43619
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786517687384214-19665-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43619
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786517687384214-19665-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10692
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786517687384214-19665-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 06:15:54 UTC on ccycloud.quasar-rnfzce.root.comops.site
build id 4581989
I20260812 06:59:22.432438 10692 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:59:22.433557 10692 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 06:59:22.443420 10698 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:59:22.443743 10692 server_base.cc:1034] running on GCE node
W20260812 06:59:22.443408 10697 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:59:22.443410 10701 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:59:22.444651 10692 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:59:22.445928 10692 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:59:22.447144 10692 hybrid_clock.cc:648] HybridClock initialized: now 1786517962447112 us; error 45 us; skew 500 ppm
Aug 12 06:59:22 dist-test-slave-65mx krb5kdc[10675](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517962, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 06:59:22.450342 10692 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260812 06:59:22.451359 10692 webserver.cc:460] Webserver started at http://127.0.0.1:36561/ using document root <none> and password file <none>
I20260812 06:59:22.452047 10692 fs_manager.cc:359] Metadata directory not provided
I20260812 06:59:22.452159 10692 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:59:22.452428 10692 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 06:59:22.454826 10692 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786517687384214-19665-0/minicluster-data/master-0/data/instance:
uuid: "f32a9bb112b24005a64d587e0843dec6"
format_stamp: "Formatted at 2026-08-12 06:59:22 on dist-test-slave-65mx"
server_key: "ba17c6ddf3b5175b397ac72166dbb2ca"
server_key_iv: "e52f218d29de980311af3d04ff613c9e"
server_key_version: "encryptionkey@0"
I20260812 06:59:22.455474 10692 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786517687384214-19665-0/minicluster-data/master-0/wal/instance:
uuid: "f32a9bb112b24005a64d587e0843dec6"
format_stamp: "Formatted at 2026-08-12 06:59:22 on dist-test-slave-65mx"
server_key: "ba17c6ddf3b5175b397ac72166dbb2ca"
server_key_iv: "e52f218d29de980311af3d04ff613c9e"
server_key_version: "encryptionkey@0"
I20260812 06:59:22.459714 10692 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.004s
I20260812 06:59:22.462788 10708 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 06:59:22.464238 10692 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260812 06:59:22.464417 10692 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786517687384214-19665-0/minicluster-data/master-0/data,/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786517687384214-19665-0/minicluster-data/master-0/wal
uuid: "f32a9bb112b24005a64d587e0843dec6"
format_stamp: "Formatted at 2026-08-12 06:59:22 on dist-test-slave-65mx"
server_key: "ba17c6ddf3b5175b397ac72166dbb2ca"
server_key_iv: "e52f218d29de980311af3d04ff613c9e"
server_key_version: "encryptionkey@0"
I20260812 06:59:22.464744 10692 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786517687384214-19665-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786517687384214-19665-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786517687384214-19665-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:59:22.470368 10692 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 06:59:22.474121 10692 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 06:59:22.474360 10692 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:59:22.482322 10692 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:43619
I20260812 06:59:22.482332 10764 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:43619 every 8 connection(s)
I20260812 06:59:22.483371 10692 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786517687384214-19665-0/minicluster-data/master-0/data/info.pb
I20260812 06:59:22.486325 10765 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 06:59:22.491936 10765 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f32a9bb112b24005a64d587e0843dec6: Bootstrap starting.
I20260812 06:59:22.491984 19665 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu as pid 10692
I20260812 06:59:22.492185 19665 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786517687384214-19665-0/minicluster-data/master-0/wal/instance
I20260812 06:59:22.492498 19665 external_mini_cluster.cc:1428] Setting key 903decf7d99f3d711350ed0b4cf198e0
I20260812 06:59:22.494752 10765 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f32a9bb112b24005a64d587e0843dec6: Neither blocks nor log segments found. Creating new log.
I20260812 06:59:22.495549 10765 log.cc:824] T 00000000000000000000000000000000 P f32a9bb112b24005a64d587e0843dec6: Log is configured to *not* fsync() on all Append() calls
I20260812 06:59:22.498132 10765 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f32a9bb112b24005a64d587e0843dec6: No bootstrap required, opened a new log
I20260812 06:59:22.498427 10768 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:59:22.493954 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37613 (local address 127.0.0.1:43619)
0812 06:59:22.494448 (+ 494us) server_negotiation.cc:206] Beginning negotiation
0812 06:59:22.494464 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0812 06:59:22.494504 (+ 40us) server_negotiation.cc:407] Connection header received
0812 06:59:22.495288 (+ 784us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:59:22.495308 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:59:22.495627 (+ 319us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:59:22.496037 (+ 410us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:59:22.496911 (+ 874us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:59:22.496917 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:59:22.496947 (+ 30us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 06:59:22.496961 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:59:22.497029 (+ 68us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 06:59:22.497057 (+ 28us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:59:22.497186 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0812 06:59:22.497265 (+ 79us) server_negotiation.cc:299] Negotiation successful
0812 06:59:22.497429 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":330,"thread_start_us":148,"threads_started":1}
I20260812 06:59:22.501411 19665 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 06:59:22.502419 10765 raft_consensus.cc:348] T 00000000000000000000000000000000 P f32a9bb112b24005a64d587e0843dec6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f32a9bb112b24005a64d587e0843dec6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43619 } }
I20260812 06:59:22.502705 10765 raft_consensus.cc:374] T 00000000000000000000000000000000 P f32a9bb112b24005a64d587e0843dec6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 06:59:22.502839 10765 raft_consensus.cc:729] T 00000000000000000000000000000000 P f32a9bb112b24005a64d587e0843dec6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f32a9bb112b24005a64d587e0843dec6, State: Initialized, Role: FOLLOWER
I20260812 06:59:22.503297 10765 consensus_queue.cc:260] T 00000000000000000000000000000000 P f32a9bb112b24005a64d587e0843dec6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f32a9bb112b24005a64d587e0843dec6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43619 } }
I20260812 06:59:22.503502 10765 raft_consensus.cc:388] T 00000000000000000000000000000000 P f32a9bb112b24005a64d587e0843dec6 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 06:59:22.503597 10765 raft_consensus.cc:482] T 00000000000000000000000000000000 P f32a9bb112b24005a64d587e0843dec6 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 06:59:22.503722 10765 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f32a9bb112b24005a64d587e0843dec6 [term 0 FOLLOWER]: Advancing to term 1
I20260812 06:59:22.505024 10765 raft_consensus.cc:504] T 00000000000000000000000000000000 P f32a9bb112b24005a64d587e0843dec6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f32a9bb112b24005a64d587e0843dec6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43619 } }
I20260812 06:59:22.505399 10765 leader_election.cc:302] T 00000000000000000000000000000000 P f32a9bb112b24005a64d587e0843dec6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f32a9bb112b24005a64d587e0843dec6; no voters:
I20260812 06:59:22.505750 10765 leader_election.cc:288] T 00000000000000000000000000000000 P f32a9bb112b24005a64d587e0843dec6 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 06:59:22.505903 10768 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:59:22.503731 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41029 (local address 127.0.0.1:43619)
0812 06:59:22.503875 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0812 06:59:22.503880 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 06:59:22.504165 (+ 285us) server_negotiation.cc:407] Connection header received
0812 06:59:22.504347 (+ 182us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:59:22.504350 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:59:22.504390 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:59:22.504491 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:59:22.505373 (+ 882us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:59:22.505376 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:59:22.505379 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 06:59:22.505392 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:59:22.505422 (+ 30us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 06:59:22.505442 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:59:22.505555 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0812 06:59:22.505722 (+ 167us) server_negotiation.cc:299] Negotiation successful
0812 06:59:22.505767 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260812 06:59:22.506013 10777 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f32a9bb112b24005a64d587e0843dec6 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 06:59:22.506271 10777 raft_consensus.cc:686] T 00000000000000000000000000000000 P f32a9bb112b24005a64d587e0843dec6 [term 1 LEADER]: Becoming Leader. State: Replica: f32a9bb112b24005a64d587e0843dec6, State: Running, Role: LEADER
I20260812 06:59:22.506660 10777 consensus_queue.cc:237] T 00000000000000000000000000000000 P f32a9bb112b24005a64d587e0843dec6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f32a9bb112b24005a64d587e0843dec6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43619 } }
I20260812 06:59:22.507346 10765 sys_catalog.cc:564] T 00000000000000000000000000000000 P f32a9bb112b24005a64d587e0843dec6 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 06:59:22.508790 10779 sys_catalog.cc:455] T 00000000000000000000000000000000 P f32a9bb112b24005a64d587e0843dec6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f32a9bb112b24005a64d587e0843dec6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f32a9bb112b24005a64d587e0843dec6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43619 } } }
I20260812 06:59:22.508962 10779 sys_catalog.cc:458] T 00000000000000000000000000000000 P f32a9bb112b24005a64d587e0843dec6 [sys.catalog]: This master's current role is: LEADER
I20260812 06:59:22.509394 10778 sys_catalog.cc:455] T 00000000000000000000000000000000 P f32a9bb112b24005a64d587e0843dec6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f32a9bb112b24005a64d587e0843dec6. Latest consensus state: current_term: 1 leader_uuid: "f32a9bb112b24005a64d587e0843dec6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f32a9bb112b24005a64d587e0843dec6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43619 } } }
I20260812 06:59:22.509531 10778 sys_catalog.cc:458] T 00000000000000000000000000000000 P f32a9bb112b24005a64d587e0843dec6 [sys.catalog]: This master's current role is: LEADER
I20260812 06:59:22.509562 10786 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 06:59:22.512840 10786 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 06:59:22.518247 10786 catalog_manager.cc:1347] Generated new cluster ID: 82c9da186a374819a053a0101c43c24e
I20260812 06:59:22.518327 10786 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 06:59:22.530174 10786 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 06:59:22.530573 10786 catalog_manager.cc:1504] Loading token signing keys...
I20260812 06:59:22.542446 10786 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f32a9bb112b24005a64d587e0843dec6: Generated new TSK 0
I20260812 06:59:22.543390 10786 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 06:59:22.551483 19665 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu with pid 10692
2026-08-12T06:59:22Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/2 (1533 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/3
Loading random data
Initializing database '/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786517687384214-19665-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 06:59:22 dist-test-slave-65mx krb5kdc[10801](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 06:59:22 dist-test-slave-65mx krb5kdc[10801](info): set up 2 sockets
Aug 12 06:59:22 dist-test-slave-65mx krb5kdc[10801](info): commencing operation
krb5kdc: starting...
W20260812 06:59:23.734390 19665 mini_kdc.cc:121] Time spent starting KDC: real 1.157s user 0.002s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786517687384214-19665-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786517687384214-19665-0/krb5kdc/test-user.keytab.
Aug 12 06:59:23 dist-test-slave-65mx krb5kdc[10801](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517963, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T06:59:23Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T06:59:23Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786517687384214-19665-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786517687384214-19665-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260812 06:59:23.876011 19665 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu
/tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786517687384214-19665-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786517687384214-19665-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786517687384214-19665-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786517687384214-19665-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39125
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786517687384214-19665-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:46551
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:39125
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786517687384214-19665-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786517687384214-19665-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786517687384214-19665-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 06:59:23.993182 10817 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:59:23.993530 10817 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:59:23.993644 10817 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 06:59:23.997816 10817 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 06:59:23.997922 10817 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 06:59:23.997962 10817 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 06:59:23.998020 10817 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 06:59:23.998061 10817 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 06:59:24.003567 10817 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:46551
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786517687384214-19665-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786517687384214-19665-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:39125
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786517687384214-19665-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39125
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786517687384214-19665-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10817
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786517687384214-19665-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 06:15:54 UTC on ccycloud.quasar-rnfzce.root.comops.site
build id 4581989
I20260812 06:59:24.004752 10817 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:59:24.005821 10817 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 06:59:24.014703 10825 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:59:24.014675 10823 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:59:24.014786 10817 server_base.cc:1034] running on GCE node
W20260812 06:59:24.014712 10822 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:59:24.015453 10817 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:59:24.016783 10817 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:59:24.017967 10817 hybrid_clock.cc:648] HybridClock initialized: now 1786517964017935 us; error 45 us; skew 500 ppm
Aug 12 06:59:24 dist-test-slave-65mx krb5kdc[10801](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517964, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 06:59:24.025146 10817 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260812 06:59:24.026216 10817 webserver.cc:460] Webserver started at http://127.0.0.1:33763/ using document root <none> and password file <none>
I20260812 06:59:24.026832 10817 fs_manager.cc:359] Metadata directory not provided
I20260812 06:59:24.026903 10817 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:59:24.027094 10817 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 06:59:24.029472 10817 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786517687384214-19665-0/minicluster-data/master-0/data/instance:
uuid: "b8e952f09c3c44f291bcd640170dd821"
format_stamp: "Formatted at 2026-08-12 06:59:24 on dist-test-slave-65mx"
server_key: "fef3fedc9ad80b83b000e900675fa87c"
server_key_iv: "6ed31fe14abf4beccb5a11f219a2a437"
server_key_version: "encryptionkey@0"
I20260812 06:59:24.030081 10817 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786517687384214-19665-0/minicluster-data/master-0/wal/instance:
uuid: "b8e952f09c3c44f291bcd640170dd821"
format_stamp: "Formatted at 2026-08-12 06:59:24 on dist-test-slave-65mx"
server_key: "fef3fedc9ad80b83b000e900675fa87c"
server_key_iv: "6ed31fe14abf4beccb5a11f219a2a437"
server_key_version: "encryptionkey@0"
I20260812 06:59:24.034577 10817 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.004s
I20260812 06:59:24.038304 10832 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 06:59:24.039690 10817 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260812 06:59:24.039847 10817 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786517687384214-19665-0/minicluster-data/master-0/data,/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786517687384214-19665-0/minicluster-data/master-0/wal
uuid: "b8e952f09c3c44f291bcd640170dd821"
format_stamp: "Formatted at 2026-08-12 06:59:24 on dist-test-slave-65mx"
server_key: "fef3fedc9ad80b83b000e900675fa87c"
server_key_iv: "6ed31fe14abf4beccb5a11f219a2a437"
server_key_version: "encryptionkey@0"
I20260812 06:59:24.039994 10817 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786517687384214-19665-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786517687384214-19665-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786517687384214-19665-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:59:24.055542 10817 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 06:59:24.059080 10817 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 06:59:24.059319 10817 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:59:24.067124 10817 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:39125
I20260812 06:59:24.067154 10889 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:39125 every 8 connection(s)
I20260812 06:59:24.068122 10817 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786517687384214-19665-0/minicluster-data/master-0/data/info.pb
I20260812 06:59:24.070935 10890 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 06:59:24.074407 19665 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu as pid 10817
I20260812 06:59:24.074626 19665 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786517687384214-19665-0/minicluster-data/master-0/wal/instance
I20260812 06:59:24.074934 19665 external_mini_cluster.cc:1428] Setting key d4d9d4f6b0f221a99a2ac32a4d758256
I20260812 06:59:24.077147 10890 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b8e952f09c3c44f291bcd640170dd821: Bootstrap starting.
I20260812 06:59:24.079698 10890 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b8e952f09c3c44f291bcd640170dd821: Neither blocks nor log segments found. Creating new log.
I20260812 06:59:24.080452 10890 log.cc:824] T 00000000000000000000000000000000 P b8e952f09c3c44f291bcd640170dd821: Log is configured to *not* fsync() on all Append() calls
I20260812 06:59:24.082692 10890 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b8e952f09c3c44f291bcd640170dd821: No bootstrap required, opened a new log
Aug 12 06:59:24 dist-test-slave-65mx krb5kdc[10801](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517963, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260812 06:59:24.086962 10890 raft_consensus.cc:348] T 00000000000000000000000000000000 P b8e952f09c3c44f291bcd640170dd821 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b8e952f09c3c44f291bcd640170dd821" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39125 } }
I20260812 06:59:24.087237 10890 raft_consensus.cc:374] T 00000000000000000000000000000000 P b8e952f09c3c44f291bcd640170dd821 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 06:59:24.087317 10890 raft_consensus.cc:729] T 00000000000000000000000000000000 P b8e952f09c3c44f291bcd640170dd821 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b8e952f09c3c44f291bcd640170dd821, State: Initialized, Role: FOLLOWER
I20260812 06:59:24.087824 10890 consensus_queue.cc:260] T 00000000000000000000000000000000 P b8e952f09c3c44f291bcd640170dd821 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b8e952f09c3c44f291bcd640170dd821" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39125 } }
I20260812 06:59:24.087978 10890 raft_consensus.cc:388] T 00000000000000000000000000000000 P b8e952f09c3c44f291bcd640170dd821 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 06:59:24.088074 10890 raft_consensus.cc:482] T 00000000000000000000000000000000 P b8e952f09c3c44f291bcd640170dd821 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 06:59:24.088179 10890 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b8e952f09c3c44f291bcd640170dd821 [term 0 FOLLOWER]: Advancing to term 1
I20260812 06:59:24.089222 10890 raft_consensus.cc:504] T 00000000000000000000000000000000 P b8e952f09c3c44f291bcd640170dd821 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b8e952f09c3c44f291bcd640170dd821" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39125 } }
I20260812 06:59:24.089581 10890 leader_election.cc:302] T 00000000000000000000000000000000 P b8e952f09c3c44f291bcd640170dd821 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b8e952f09c3c44f291bcd640170dd821; no voters:
I20260812 06:59:24.089979 10890 leader_election.cc:288] T 00000000000000000000000000000000 P b8e952f09c3c44f291bcd640170dd821 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 06:59:24.090078 10895 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b8e952f09c3c44f291bcd640170dd821 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 06:59:24.090309 10895 raft_consensus.cc:686] T 00000000000000000000000000000000 P b8e952f09c3c44f291bcd640170dd821 [term 1 LEADER]: Becoming Leader. State: Replica: b8e952f09c3c44f291bcd640170dd821, State: Running, Role: LEADER
I20260812 06:59:24.090672 10895 consensus_queue.cc:237] T 00000000000000000000000000000000 P b8e952f09c3c44f291bcd640170dd821 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b8e952f09c3c44f291bcd640170dd821" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39125 } }
I20260812 06:59:24.091414 10893 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:59:24.076507 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60057 (local address 127.0.0.1:39125)
0812 06:59:24.077082 (+ 575us) server_negotiation.cc:206] Beginning negotiation
0812 06:59:24.077102 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0812 06:59:24.077141 (+ 39us) server_negotiation.cc:407] Connection header received
0812 06:59:24.077883 (+ 742us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:59:24.077901 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:59:24.078258 (+ 357us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:59:24.078560 (+ 302us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:59:24.079519 (+ 959us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:59:24.080819 (+ 1300us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:59:24.081954 (+ 1135us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:59:24.082472 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:59:24.082612 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:59:24.085684 (+ 3072us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:59:24.085691 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:59:24.085706 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:59:24.085739 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:59:24.087852 (+ 2113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:59:24.088475 (+ 623us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:59:24.088487 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:59:24.088492 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:59:24.088687 (+ 195us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:59:24.089010 (+ 323us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:59:24.089014 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:59:24.089016 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:59:24.089522 (+ 506us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:59:24.089743 (+ 221us) server_negotiation.cc:1035] Waiting for connection context
0812 06:59:24.090091 (+ 348us) server_negotiation.cc:299] Negotiation successful
0812 06:59:24.090474 (+ 383us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":372,"thread_start_us":161,"threads_started":1}
I20260812 06:59:24.091881 10897 sys_catalog.cc:455] T 00000000000000000000000000000000 P b8e952f09c3c44f291bcd640170dd821 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b8e952f09c3c44f291bcd640170dd821. Latest consensus state: current_term: 1 leader_uuid: "b8e952f09c3c44f291bcd640170dd821" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b8e952f09c3c44f291bcd640170dd821" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39125 } } }
I20260812 06:59:24.092031 10897 sys_catalog.cc:458] T 00000000000000000000000000000000 P b8e952f09c3c44f291bcd640170dd821 [sys.catalog]: This master's current role is: LEADER
I20260812 06:59:24.091899 10896 sys_catalog.cc:455] T 00000000000000000000000000000000 P b8e952f09c3c44f291bcd640170dd821 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b8e952f09c3c44f291bcd640170dd821" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b8e952f09c3c44f291bcd640170dd821" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39125 } } }
I20260812 06:59:24.092162 10896 sys_catalog.cc:458] T 00000000000000000000000000000000 P b8e952f09c3c44f291bcd640170dd821 [sys.catalog]: This master's current role is: LEADER
I20260812 06:59:24.092506 10899 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 06:59:24.091427 10890 sys_catalog.cc:564] T 00000000000000000000000000000000 P b8e952f09c3c44f291bcd640170dd821 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 06:59:24.095343 19665 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 06:59:24.096334 10899 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 06:59:24.103538 10899 catalog_manager.cc:1347] Generated new cluster ID: cc25ab4aedfd4ddf9a64a680795e0fed
I20260812 06:59:24.103664 10899 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 06:59:24.107419 10893 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:59:24.097975 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38155 (local address 127.0.0.1:39125)
0812 06:59:24.098096 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0812 06:59:24.098101 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 06:59:24.098127 (+ 26us) server_negotiation.cc:407] Connection header received
0812 06:59:24.098174 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:59:24.098177 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:59:24.098222 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:59:24.098322 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:59:24.099263 (+ 941us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:59:24.100088 (+ 825us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:59:24.101239 (+ 1151us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:59:24.101692 (+ 453us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:59:24.101828 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:59:24.103482 (+ 1654us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:59:24.103486 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:59:24.103490 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:59:24.103520 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:59:24.105566 (+ 2046us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:59:24.106099 (+ 533us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:59:24.106102 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:59:24.106104 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:59:24.106193 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:59:24.106506 (+ 313us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:59:24.106509 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:59:24.106511 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:59:24.106717 (+ 206us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:59:24.106837 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0812 06:59:24.107125 (+ 288us) server_negotiation.cc:299] Negotiation successful
0812 06:59:24.107248 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260812 06:59:24.130036 10899 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 06:59:24.131090 10899 catalog_manager.cc:1504] Loading token signing keys...
I20260812 06:59:24.137705 10899 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b8e952f09c3c44f291bcd640170dd821: Generated new TSK 0
I20260812 06:59:24.138576 10899 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 06:59:24.146027 19665 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu with pid 10817
2026-08-12T06:59:24Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/3 (1593 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/4
Loading random data
Initializing database '/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786517687384214-19665-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 06:59:24 dist-test-slave-65mx krb5kdc[10925](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 06:59:24 dist-test-slave-65mx krb5kdc[10925](info): set up 2 sockets
Aug 12 06:59:24 dist-test-slave-65mx krb5kdc[10925](info): commencing operation
krb5kdc: starting...
W20260812 06:59:25.300794 19665 mini_kdc.cc:121] Time spent starting KDC: real 1.129s user 0.000s sys 0.009s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786517687384214-19665-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786517687384214-19665-0/krb5kdc/test-user.keytab.
Aug 12 06:59:25 dist-test-slave-65mx krb5kdc[10925](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517965, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T06:59:25Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T06:59:25Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786517687384214-19665-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786517687384214-19665-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260812 06:59:25.445144 19665 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu
/tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786517687384214-19665-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786517687384214-19665-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786517687384214-19665-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786517687384214-19665-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45807
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786517687384214-19665-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:36143
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:45807
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786517687384214-19665-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786517687384214-19665-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786517687384214-19665-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 06:59:25.567690 10939 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:59:25.568010 10939 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:59:25.568056 10939 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 06:59:25.572371 10939 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 06:59:25.572450 10939 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 06:59:25.572471 10939 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 06:59:25.572489 10939 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 06:59:25.572506 10939 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 06:59:25.578183 10939 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:36143
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786517687384214-19665-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786517687384214-19665-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:45807
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786517687384214-19665-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45807
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786517687384214-19665-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10939
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786517687384214-19665-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 06:15:54 UTC on ccycloud.quasar-rnfzce.root.comops.site
build id 4581989
I20260812 06:59:25.579376 10939 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:59:25.580459 10939 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 06:59:25.589424 10950 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:59:25.589632 10939 server_base.cc:1034] running on GCE node
W20260812 06:59:25.589407 10947 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:59:25.589406 10948 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:59:25.590334 10939 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:59:25.591569 10939 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:59:25.592774 10939 hybrid_clock.cc:648] HybridClock initialized: now 1786517965592754 us; error 33 us; skew 500 ppm
Aug 12 06:59:25 dist-test-slave-65mx krb5kdc[10925](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517965, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 06:59:25.596091 10939 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260812 06:59:25.597258 10939 webserver.cc:460] Webserver started at http://127.0.0.1:43129/ using document root <none> and password file <none>
I20260812 06:59:25.597877 10939 fs_manager.cc:359] Metadata directory not provided
I20260812 06:59:25.597993 10939 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:59:25.598241 10939 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 06:59:25.600459 10939 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786517687384214-19665-0/minicluster-data/master-0/data/instance:
uuid: "bca474a4b1f64af2a3659d4613aaa82b"
format_stamp: "Formatted at 2026-08-12 06:59:25 on dist-test-slave-65mx"
server_key: "f452d14c4e4eb2ac909822aea2a66446"
server_key_iv: "cd49c60b48fcb0fe8de7a03405610d46"
server_key_version: "encryptionkey@0"
I20260812 06:59:25.601240 10939 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786517687384214-19665-0/minicluster-data/master-0/wal/instance:
uuid: "bca474a4b1f64af2a3659d4613aaa82b"
format_stamp: "Formatted at 2026-08-12 06:59:25 on dist-test-slave-65mx"
server_key: "f452d14c4e4eb2ac909822aea2a66446"
server_key_iv: "cd49c60b48fcb0fe8de7a03405610d46"
server_key_version: "encryptionkey@0"
I20260812 06:59:25.605849 10939 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260812 06:59:25.608986 10957 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 06:59:25.610526 10939 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 06:59:25.610702 10939 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786517687384214-19665-0/minicluster-data/master-0/data,/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786517687384214-19665-0/minicluster-data/master-0/wal
uuid: "bca474a4b1f64af2a3659d4613aaa82b"
format_stamp: "Formatted at 2026-08-12 06:59:25 on dist-test-slave-65mx"
server_key: "f452d14c4e4eb2ac909822aea2a66446"
server_key_iv: "cd49c60b48fcb0fe8de7a03405610d46"
server_key_version: "encryptionkey@0"
I20260812 06:59:25.610852 10939 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786517687384214-19665-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786517687384214-19665-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786517687384214-19665-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:59:25.635651 10939 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 06:59:25.639595 10939 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 06:59:25.639855 10939 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:59:25.648077 10939 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:45807
I20260812 06:59:25.648101 11014 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:45807 every 8 connection(s)
I20260812 06:59:25.649277 10939 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786517687384214-19665-0/minicluster-data/master-0/data/info.pb
I20260812 06:59:25.652396 11015 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 06:59:25.653779 19665 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu as pid 10939
I20260812 06:59:25.653955 19665 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786517687384214-19665-0/minicluster-data/master-0/wal/instance
I20260812 06:59:25.654271 19665 external_mini_cluster.cc:1428] Setting key de78fb6664649886bab20884888c4e6c
I20260812 06:59:25.659416 11015 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bca474a4b1f64af2a3659d4613aaa82b: Bootstrap starting.
I20260812 06:59:25.662410 11015 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P bca474a4b1f64af2a3659d4613aaa82b: Neither blocks nor log segments found. Creating new log.
I20260812 06:59:25.663305 11015 log.cc:824] T 00000000000000000000000000000000 P bca474a4b1f64af2a3659d4613aaa82b: Log is configured to *not* fsync() on all Append() calls
I20260812 06:59:25.664923 11018 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:59:25.656083 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35793 (local address 127.0.0.1:45807)
0812 06:59:25.656628 (+ 545us) server_negotiation.cc:206] Beginning negotiation
0812 06:59:25.656646 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0812 06:59:25.656684 (+ 38us) server_negotiation.cc:407] Connection header received
0812 06:59:25.657401 (+ 717us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:59:25.657420 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:59:25.657780 (+ 360us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:59:25.658077 (+ 297us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:59:25.658609 (+ 532us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:59:25.659685 (+ 1076us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:59:25.660979 (+ 1294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:59:25.661489 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:59:25.661629 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:59:25.662620 (+ 991us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:59:25.662626 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:59:25.662642 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 06:59:25.663056 (+ 414us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:59:25.663115 (+ 59us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 06:59:25.663160 (+ 45us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:59:25.663247 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0812 06:59:25.663455 (+ 208us) server_negotiation.cc:299] Negotiation successful
0812 06:59:25.663981 (+ 526us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":327,"thread_start_us":165,"threads_started":1}
I20260812 06:59:25.665730 11015 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bca474a4b1f64af2a3659d4613aaa82b: No bootstrap required, opened a new log
I20260812 06:59:25.667917 19665 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 06:59:25.670272 11015 raft_consensus.cc:348] T 00000000000000000000000000000000 P bca474a4b1f64af2a3659d4613aaa82b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bca474a4b1f64af2a3659d4613aaa82b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45807 } }
I20260812 06:59:25.670606 11015 raft_consensus.cc:374] T 00000000000000000000000000000000 P bca474a4b1f64af2a3659d4613aaa82b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 06:59:25.670686 11015 raft_consensus.cc:729] T 00000000000000000000000000000000 P bca474a4b1f64af2a3659d4613aaa82b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bca474a4b1f64af2a3659d4613aaa82b, State: Initialized, Role: FOLLOWER
I20260812 06:59:25.671224 11015 consensus_queue.cc:260] T 00000000000000000000000000000000 P bca474a4b1f64af2a3659d4613aaa82b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bca474a4b1f64af2a3659d4613aaa82b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45807 } }
I20260812 06:59:25.671407 11015 raft_consensus.cc:388] T 00000000000000000000000000000000 P bca474a4b1f64af2a3659d4613aaa82b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 06:59:25.671476 11015 raft_consensus.cc:482] T 00000000000000000000000000000000 P bca474a4b1f64af2a3659d4613aaa82b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 06:59:25.671593 11015 raft_consensus.cc:3037] T 00000000000000000000000000000000 P bca474a4b1f64af2a3659d4613aaa82b [term 0 FOLLOWER]: Advancing to term 1
I20260812 06:59:25.672878 11015 raft_consensus.cc:504] T 00000000000000000000000000000000 P bca474a4b1f64af2a3659d4613aaa82b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bca474a4b1f64af2a3659d4613aaa82b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45807 } }
I20260812 06:59:25.673285 11015 leader_election.cc:302] T 00000000000000000000000000000000 P bca474a4b1f64af2a3659d4613aaa82b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: bca474a4b1f64af2a3659d4613aaa82b; no voters:
I20260812 06:59:25.673661 11015 leader_election.cc:288] T 00000000000000000000000000000000 P bca474a4b1f64af2a3659d4613aaa82b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 06:59:25.673830 11027 raft_consensus.cc:2781] T 00000000000000000000000000000000 P bca474a4b1f64af2a3659d4613aaa82b [term 1 FOLLOWER]: Leader election won for term 1
I20260812 06:59:25.674106 11027 raft_consensus.cc:686] T 00000000000000000000000000000000 P bca474a4b1f64af2a3659d4613aaa82b [term 1 LEADER]: Becoming Leader. State: Replica: bca474a4b1f64af2a3659d4613aaa82b, State: Running, Role: LEADER
I20260812 06:59:25.674472 11027 consensus_queue.cc:237] T 00000000000000000000000000000000 P bca474a4b1f64af2a3659d4613aaa82b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bca474a4b1f64af2a3659d4613aaa82b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45807 } }
I20260812 06:59:25.675204 11018 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:59:25.670442 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51755 (local address 127.0.0.1:45807)
0812 06:59:25.670571 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0812 06:59:25.670578 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 06:59:25.670735 (+ 157us) server_negotiation.cc:407] Connection header received
0812 06:59:25.670888 (+ 153us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:59:25.670890 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:59:25.670976 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:59:25.671098 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:59:25.671489 (+ 391us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:59:25.672130 (+ 641us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:59:25.673261 (+ 1131us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:59:25.673607 (+ 346us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:59:25.673738 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:59:25.674701 (+ 963us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:59:25.674704 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:59:25.674707 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 06:59:25.674721 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:59:25.674752 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 06:59:25.674765 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:59:25.674836 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0812 06:59:25.675003 (+ 167us) server_negotiation.cc:299] Negotiation successful
0812 06:59:25.675062 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260812 06:59:25.675189 11015 sys_catalog.cc:564] T 00000000000000000000000000000000 P bca474a4b1f64af2a3659d4613aaa82b [sys.catalog]: configured and running, proceeding with master startup.
I20260812 06:59:25.675848 11030 sys_catalog.cc:455] T 00000000000000000000000000000000 P bca474a4b1f64af2a3659d4613aaa82b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "bca474a4b1f64af2a3659d4613aaa82b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bca474a4b1f64af2a3659d4613aaa82b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45807 } } }
I20260812 06:59:25.675973 11030 sys_catalog.cc:458] T 00000000000000000000000000000000 P bca474a4b1f64af2a3659d4613aaa82b [sys.catalog]: This master's current role is: LEADER
I20260812 06:59:25.676354 11029 sys_catalog.cc:455] T 00000000000000000000000000000000 P bca474a4b1f64af2a3659d4613aaa82b [sys.catalog]: SysCatalogTable state changed. Reason: New leader bca474a4b1f64af2a3659d4613aaa82b. Latest consensus state: current_term: 1 leader_uuid: "bca474a4b1f64af2a3659d4613aaa82b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bca474a4b1f64af2a3659d4613aaa82b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45807 } } }
I20260812 06:59:25.676492 11029 sys_catalog.cc:458] T 00000000000000000000000000000000 P bca474a4b1f64af2a3659d4613aaa82b [sys.catalog]: This master's current role is: LEADER
W20260812 06:59:25.680394 11044 catalog_manager.cc:1558] T 00000000000000000000000000000000 P bca474a4b1f64af2a3659d4613aaa82b: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 06:59:25.680496 11044 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 06:59:25.680632 11033 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 06:59:25.681412 11033 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 06:59:25.687633 11033 catalog_manager.cc:1347] Generated new cluster ID: 62449944ffbb4abeb9fa8a08140d40e4
I20260812 06:59:25.687741 11033 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 06:59:25.693768 11033 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 06:59:25.694849 11033 catalog_manager.cc:1504] Loading token signing keys...
I20260812 06:59:25.699950 11033 catalog_manager.cc:5868] T 00000000000000000000000000000000 P bca474a4b1f64af2a3659d4613aaa82b: Generated new TSK 0
I20260812 06:59:25.700742 11033 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 06:59:25.707055 19665 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu with pid 10939
2026-08-12T06:59:25Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/4 (1557 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/5
Loading random data
Initializing database '/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786517687384214-19665-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 06:59:25 dist-test-slave-65mx krb5kdc[11051](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 06:59:25 dist-test-slave-65mx krb5kdc[11051](info): set up 2 sockets
Aug 12 06:59:25 dist-test-slave-65mx krb5kdc[11051](info): commencing operation
krb5kdc: starting...
W20260812 06:59:26.832527 19665 mini_kdc.cc:121] Time spent starting KDC: real 1.103s user 0.002s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786517687384214-19665-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786517687384214-19665-0/krb5kdc/test-user.keytab.
Aug 12 06:59:26 dist-test-slave-65mx krb5kdc[11051](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517966, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T06:59:26Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T06:59:26Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786517687384214-19665-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786517687384214-19665-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260812 06:59:26.976517 19665 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu
/tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786517687384214-19665-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786517687384214-19665-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786517687384214-19665-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786517687384214-19665-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35293
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786517687384214-19665-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:43917
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:35293
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786517687384214-19665-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786517687384214-19665-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786517687384214-19665-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 06:59:27.093883 11065 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:59:27.094245 11065 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:59:27.094332 11065 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 06:59:27.098596 11065 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 06:59:27.098713 11065 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 06:59:27.098752 11065 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 06:59:27.098793 11065 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 06:59:27.098827 11065 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 06:59:27.105592 11065 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:43917
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786517687384214-19665-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786517687384214-19665-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:35293
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786517687384214-19665-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35293
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786517687384214-19665-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11065
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786517687384214-19665-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 06:15:54 UTC on ccycloud.quasar-rnfzce.root.comops.site
build id 4581989
I20260812 06:59:27.106827 11065 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:59:27.107923 11065 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 06:59:27.117112 11072 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:59:27.117466 11065 server_base.cc:1034] running on GCE node
W20260812 06:59:27.117116 11073 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:59:27.117313 11075 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:59:27.118216 11065 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:59:27.119453 11065 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:59:27.120651 11065 hybrid_clock.cc:648] HybridClock initialized: now 1786517967120631 us; error 33 us; skew 500 ppm
Aug 12 06:59:27 dist-test-slave-65mx krb5kdc[11051](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517967, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 06:59:27.123760 11065 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260812 06:59:27.124840 11065 webserver.cc:460] Webserver started at http://127.0.0.1:34045/ using document root <none> and password file <none>
I20260812 06:59:27.125423 11065 fs_manager.cc:359] Metadata directory not provided
I20260812 06:59:27.125521 11065 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:59:27.125763 11065 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 06:59:27.127890 11065 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786517687384214-19665-0/minicluster-data/master-0/data/instance:
uuid: "fff82e5e64844dfebabea36ac2f609e7"
format_stamp: "Formatted at 2026-08-12 06:59:27 on dist-test-slave-65mx"
server_key: "01b8506f60f71587c965ef98231d3168"
server_key_iv: "1625a620309631d1e4f7808c722f3884"
server_key_version: "encryptionkey@0"
I20260812 06:59:27.128587 11065 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786517687384214-19665-0/minicluster-data/master-0/wal/instance:
uuid: "fff82e5e64844dfebabea36ac2f609e7"
format_stamp: "Formatted at 2026-08-12 06:59:27 on dist-test-slave-65mx"
server_key: "01b8506f60f71587c965ef98231d3168"
server_key_iv: "1625a620309631d1e4f7808c722f3884"
server_key_version: "encryptionkey@0"
I20260812 06:59:27.132666 11065 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260812 06:59:27.135426 11082 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 06:59:27.136902 11065 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.003s
I20260812 06:59:27.137081 11065 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786517687384214-19665-0/minicluster-data/master-0/data,/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786517687384214-19665-0/minicluster-data/master-0/wal
uuid: "fff82e5e64844dfebabea36ac2f609e7"
format_stamp: "Formatted at 2026-08-12 06:59:27 on dist-test-slave-65mx"
server_key: "01b8506f60f71587c965ef98231d3168"
server_key_iv: "1625a620309631d1e4f7808c722f3884"
server_key_version: "encryptionkey@0"
I20260812 06:59:27.137220 11065 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786517687384214-19665-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786517687384214-19665-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786517687384214-19665-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:59:27.142236 11065 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 06:59:27.145972 11065 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 06:59:27.146198 11065 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:59:27.154556 11065 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:35293
I20260812 06:59:27.154620 11147 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:35293 every 8 connection(s)
I20260812 06:59:27.155632 11065 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786517687384214-19665-0/minicluster-data/master-0/data/info.pb
I20260812 06:59:27.158597 11148 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 06:59:27.165139 19665 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu as pid 11065
I20260812 06:59:27.165318 19665 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786517687384214-19665-0/minicluster-data/master-0/wal/instance
I20260812 06:59:27.165344 11148 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fff82e5e64844dfebabea36ac2f609e7: Bootstrap starting.
I20260812 06:59:27.165710 19665 external_mini_cluster.cc:1428] Setting key 2b927a454add3fade34fc5b209371b42
I20260812 06:59:27.168169 11148 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P fff82e5e64844dfebabea36ac2f609e7: Neither blocks nor log segments found. Creating new log.
I20260812 06:59:27.169042 11148 log.cc:824] T 00000000000000000000000000000000 P fff82e5e64844dfebabea36ac2f609e7: Log is configured to *not* fsync() on all Append() calls
I20260812 06:59:27.171342 11148 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fff82e5e64844dfebabea36ac2f609e7: No bootstrap required, opened a new log
I20260812 06:59:27.172124 11151 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:59:27.167302 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46973 (local address 127.0.0.1:35293)
0812 06:59:27.167936 (+ 634us) server_negotiation.cc:206] Beginning negotiation
0812 06:59:27.167954 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0812 06:59:27.168009 (+ 55us) server_negotiation.cc:407] Connection header received
0812 06:59:27.168913 (+ 904us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:59:27.168936 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:59:27.169312 (+ 376us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:59:27.169701 (+ 389us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:59:27.170574 (+ 873us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:59:27.170579 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:59:27.170591 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 06:59:27.170602 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:59:27.170657 (+ 55us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 06:59:27.170677 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:59:27.170814 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0812 06:59:27.170926 (+ 112us) server_negotiation.cc:299] Negotiation successful
0812 06:59:27.171110 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":439,"thread_start_us":272,"threads_started":1}
I20260812 06:59:27.175767 19665 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 06:59:27.175753 11148 raft_consensus.cc:348] T 00000000000000000000000000000000 P fff82e5e64844dfebabea36ac2f609e7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fff82e5e64844dfebabea36ac2f609e7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35293 } }
I20260812 06:59:27.176050 11148 raft_consensus.cc:374] T 00000000000000000000000000000000 P fff82e5e64844dfebabea36ac2f609e7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 06:59:27.176091 11148 raft_consensus.cc:729] T 00000000000000000000000000000000 P fff82e5e64844dfebabea36ac2f609e7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fff82e5e64844dfebabea36ac2f609e7, State: Initialized, Role: FOLLOWER
I20260812 06:59:27.176599 11148 consensus_queue.cc:260] T 00000000000000000000000000000000 P fff82e5e64844dfebabea36ac2f609e7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fff82e5e64844dfebabea36ac2f609e7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35293 } }
I20260812 06:59:27.176749 11148 raft_consensus.cc:388] T 00000000000000000000000000000000 P fff82e5e64844dfebabea36ac2f609e7 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 06:59:27.176836 11148 raft_consensus.cc:482] T 00000000000000000000000000000000 P fff82e5e64844dfebabea36ac2f609e7 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 06:59:27.176949 11148 raft_consensus.cc:3037] T 00000000000000000000000000000000 P fff82e5e64844dfebabea36ac2f609e7 [term 0 FOLLOWER]: Advancing to term 1
I20260812 06:59:27.178094 11148 raft_consensus.cc:504] T 00000000000000000000000000000000 P fff82e5e64844dfebabea36ac2f609e7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fff82e5e64844dfebabea36ac2f609e7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35293 } }
I20260812 06:59:27.178514 11148 leader_election.cc:302] T 00000000000000000000000000000000 P fff82e5e64844dfebabea36ac2f609e7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: fff82e5e64844dfebabea36ac2f609e7; no voters:
I20260812 06:59:27.178908 11148 leader_election.cc:288] T 00000000000000000000000000000000 P fff82e5e64844dfebabea36ac2f609e7 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 06:59:27.178982 11158 raft_consensus.cc:2781] T 00000000000000000000000000000000 P fff82e5e64844dfebabea36ac2f609e7 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 06:59:27.179193 11158 raft_consensus.cc:686] T 00000000000000000000000000000000 P fff82e5e64844dfebabea36ac2f609e7 [term 1 LEADER]: Becoming Leader. State: Replica: fff82e5e64844dfebabea36ac2f609e7, State: Running, Role: LEADER
I20260812 06:59:27.179543 11158 consensus_queue.cc:237] T 00000000000000000000000000000000 P fff82e5e64844dfebabea36ac2f609e7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fff82e5e64844dfebabea36ac2f609e7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35293 } }
I20260812 06:59:27.180738 11148 sys_catalog.cc:564] T 00000000000000000000000000000000 P fff82e5e64844dfebabea36ac2f609e7 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 06:59:27.181195 11159 sys_catalog.cc:455] T 00000000000000000000000000000000 P fff82e5e64844dfebabea36ac2f609e7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader fff82e5e64844dfebabea36ac2f609e7. Latest consensus state: current_term: 1 leader_uuid: "fff82e5e64844dfebabea36ac2f609e7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fff82e5e64844dfebabea36ac2f609e7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35293 } } }
I20260812 06:59:27.181314 11159 sys_catalog.cc:458] T 00000000000000000000000000000000 P fff82e5e64844dfebabea36ac2f609e7 [sys.catalog]: This master's current role is: LEADER
I20260812 06:59:27.181660 11151 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:59:27.178603 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45681 (local address 127.0.0.1:35293)
0812 06:59:27.178748 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0812 06:59:27.178754 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 06:59:27.179026 (+ 272us) server_negotiation.cc:407] Connection header received
0812 06:59:27.179269 (+ 243us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:59:27.179273 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:59:27.179486 (+ 213us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:59:27.179586 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:59:27.180725 (+ 1139us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:59:27.180729 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:59:27.180732 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 06:59:27.180746 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:59:27.180776 (+ 30us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 06:59:27.180789 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:59:27.180873 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0812 06:59:27.181238 (+ 365us) server_negotiation.cc:299] Negotiation successful
0812 06:59:27.181286 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260812 06:59:27.182089 11160 sys_catalog.cc:455] T 00000000000000000000000000000000 P fff82e5e64844dfebabea36ac2f609e7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "fff82e5e64844dfebabea36ac2f609e7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fff82e5e64844dfebabea36ac2f609e7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35293 } } }
I20260812 06:59:27.182188 11160 sys_catalog.cc:458] T 00000000000000000000000000000000 P fff82e5e64844dfebabea36ac2f609e7 [sys.catalog]: This master's current role is: LEADER
I20260812 06:59:27.183145 11162 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 06:59:27.186817 11162 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 06:59:27.192799 11162 catalog_manager.cc:1347] Generated new cluster ID: 8ea2ce6014b245f3b9206dfd27765d0a
I20260812 06:59:27.192896 11162 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 06:59:27.211925 11162 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 06:59:27.212325 11162 catalog_manager.cc:1504] Loading token signing keys...
I20260812 06:59:27.233153 11162 catalog_manager.cc:5868] T 00000000000000000000000000000000 P fff82e5e64844dfebabea36ac2f609e7: Generated new TSK 0
I20260812 06:59:27.234127 11162 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 06:59:27.242871 19665 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu with pid 11065
2026-08-12T06:59:27Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/5 (1537 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/6
Loading random data
Initializing database '/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786517687384214-19665-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 06:59:27 dist-test-slave-65mx krb5kdc[11181](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 06:59:27 dist-test-slave-65mx krb5kdc[11181](info): set up 2 sockets
Aug 12 06:59:27 dist-test-slave-65mx krb5kdc[11181](info): commencing operation
krb5kdc: starting...
W20260812 06:59:28.420897 19665 mini_kdc.cc:121] Time spent starting KDC: real 1.154s user 0.004s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786517687384214-19665-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786517687384214-19665-0/krb5kdc/test-user.keytab.
Aug 12 06:59:28 dist-test-slave-65mx krb5kdc[11181](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517968, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T06:59:28Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T06:59:28Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786517687384214-19665-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786517687384214-19665-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260812 06:59:28.563745 19665 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu
/tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786517687384214-19665-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786517687384214-19665-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786517687384214-19665-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786517687384214-19665-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34239
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786517687384214-19665-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:35487
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:34239
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786517687384214-19665-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786517687384214-19665-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786517687384214-19665-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 06:59:28.688149 11194 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:59:28.688457 11194 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:59:28.688501 11194 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 06:59:28.692843 11194 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 06:59:28.692924 11194 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 06:59:28.692945 11194 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 06:59:28.692963 11194 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 06:59:28.693102 11194 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 06:59:28.698774 11194 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:35487
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786517687384214-19665-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786517687384214-19665-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:34239
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786517687384214-19665-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34239
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786517687384214-19665-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11194
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786517687384214-19665-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 06:15:54 UTC on ccycloud.quasar-rnfzce.root.comops.site
build id 4581989
I20260812 06:59:28.699998 11194 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:59:28.701107 11194 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 06:59:28.710095 11200 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:59:28.710052 11202 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:59:28.710039 11199 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:59:28.710265 11194 server_base.cc:1034] running on GCE node
I20260812 06:59:28.710906 11194 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:59:28.712118 11194 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:59:28.713311 11194 hybrid_clock.cc:648] HybridClock initialized: now 1786517968713273 us; error 50 us; skew 500 ppm
Aug 12 06:59:28 dist-test-slave-65mx krb5kdc[11181](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517968, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 06:59:28.716325 11194 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260812 06:59:28.717401 11194 webserver.cc:460] Webserver started at http://127.0.0.1:46151/ using document root <none> and password file <none>
I20260812 06:59:28.718005 11194 fs_manager.cc:359] Metadata directory not provided
I20260812 06:59:28.718104 11194 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:59:28.718353 11194 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 06:59:28.720503 11194 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786517687384214-19665-0/minicluster-data/master-0/data/instance:
uuid: "5085ebf85cb446d7a5d8cff76edb90a1"
format_stamp: "Formatted at 2026-08-12 06:59:28 on dist-test-slave-65mx"
server_key: "861db7dc1fe8d035e14de167e156b6f4"
server_key_iv: "c6e4bd412264d7367d3f71c6a0fb1666"
server_key_version: "encryptionkey@0"
I20260812 06:59:28.721168 11194 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786517687384214-19665-0/minicluster-data/master-0/wal/instance:
uuid: "5085ebf85cb446d7a5d8cff76edb90a1"
format_stamp: "Formatted at 2026-08-12 06:59:28 on dist-test-slave-65mx"
server_key: "861db7dc1fe8d035e14de167e156b6f4"
server_key_iv: "c6e4bd412264d7367d3f71c6a0fb1666"
server_key_version: "encryptionkey@0"
I20260812 06:59:28.725267 11194 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260812 06:59:28.728052 11212 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 06:59:28.729501 11194 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260812 06:59:28.729683 11194 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786517687384214-19665-0/minicluster-data/master-0/data,/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786517687384214-19665-0/minicluster-data/master-0/wal
uuid: "5085ebf85cb446d7a5d8cff76edb90a1"
format_stamp: "Formatted at 2026-08-12 06:59:28 on dist-test-slave-65mx"
server_key: "861db7dc1fe8d035e14de167e156b6f4"
server_key_iv: "c6e4bd412264d7367d3f71c6a0fb1666"
server_key_version: "encryptionkey@0"
I20260812 06:59:28.729818 11194 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786517687384214-19665-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786517687384214-19665-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786517687384214-19665-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:59:28.750548 11194 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 06:59:28.754334 11194 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 06:59:28.754724 11194 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:59:28.763684 11194 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:34239
I20260812 06:59:28.763679 11278 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:34239 every 8 connection(s)
I20260812 06:59:28.764832 11194 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786517687384214-19665-0/minicluster-data/master-0/data/info.pb
I20260812 06:59:28.767870 11279 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 06:59:28.772838 19665 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu as pid 11194
I20260812 06:59:28.773026 19665 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786517687384214-19665-0/minicluster-data/master-0/wal/instance
I20260812 06:59:28.773334 19665 external_mini_cluster.cc:1428] Setting key ac379df635c2fa1fcb67cb4dcb7c9cde
I20260812 06:59:28.774315 11279 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5085ebf85cb446d7a5d8cff76edb90a1: Bootstrap starting.
I20260812 06:59:28.777114 11279 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5085ebf85cb446d7a5d8cff76edb90a1: Neither blocks nor log segments found. Creating new log.
I20260812 06:59:28.777902 11279 log.cc:824] T 00000000000000000000000000000000 P 5085ebf85cb446d7a5d8cff76edb90a1: Log is configured to *not* fsync() on all Append() calls
I20260812 06:59:28.780030 11279 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5085ebf85cb446d7a5d8cff76edb90a1: No bootstrap required, opened a new log
Aug 12 06:59:28 dist-test-slave-65mx krb5kdc[11181](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517968, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260812 06:59:28.784531 11279 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5085ebf85cb446d7a5d8cff76edb90a1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5085ebf85cb446d7a5d8cff76edb90a1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34239 } }
I20260812 06:59:28.784937 11279 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5085ebf85cb446d7a5d8cff76edb90a1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 06:59:28.785006 11279 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5085ebf85cb446d7a5d8cff76edb90a1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5085ebf85cb446d7a5d8cff76edb90a1, State: Initialized, Role: FOLLOWER
I20260812 06:59:28.785588 11279 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5085ebf85cb446d7a5d8cff76edb90a1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5085ebf85cb446d7a5d8cff76edb90a1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34239 } }
I20260812 06:59:28.785754 11279 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5085ebf85cb446d7a5d8cff76edb90a1 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 06:59:28.785825 11279 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5085ebf85cb446d7a5d8cff76edb90a1 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 06:59:28.785920 11279 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5085ebf85cb446d7a5d8cff76edb90a1 [term 0 FOLLOWER]: Advancing to term 1
I20260812 06:59:28.787158 11279 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5085ebf85cb446d7a5d8cff76edb90a1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5085ebf85cb446d7a5d8cff76edb90a1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34239 } }
I20260812 06:59:28.787560 11279 leader_election.cc:302] T 00000000000000000000000000000000 P 5085ebf85cb446d7a5d8cff76edb90a1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5085ebf85cb446d7a5d8cff76edb90a1; no voters:
I20260812 06:59:28.787945 11279 leader_election.cc:288] T 00000000000000000000000000000000 P 5085ebf85cb446d7a5d8cff76edb90a1 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 06:59:28.788120 11284 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5085ebf85cb446d7a5d8cff76edb90a1 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 06:59:28.788388 11284 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5085ebf85cb446d7a5d8cff76edb90a1 [term 1 LEADER]: Becoming Leader. State: Replica: 5085ebf85cb446d7a5d8cff76edb90a1, State: Running, Role: LEADER
I20260812 06:59:28.788761 11284 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5085ebf85cb446d7a5d8cff76edb90a1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5085ebf85cb446d7a5d8cff76edb90a1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34239 } }
I20260812 06:59:28.789394 11279 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5085ebf85cb446d7a5d8cff76edb90a1 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 06:59:28.790434 11282 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:59:28.774992 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.40:60691 (local address 127.0.0.1:34239)
0812 06:59:28.775647 (+ 655us) server_negotiation.cc:206] Beginning negotiation
0812 06:59:28.775670 (+ 23us) server_negotiation.cc:399] Waiting for connection header
0812 06:59:28.775710 (+ 40us) server_negotiation.cc:407] Connection header received
0812 06:59:28.776406 (+ 696us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:59:28.776423 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:59:28.776876 (+ 453us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:59:28.777233 (+ 357us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:59:28.778104 (+ 871us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:59:28.779334 (+ 1230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:59:28.780245 (+ 911us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:59:28.780969 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:59:28.781146 (+ 177us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:59:28.784044 (+ 2898us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:59:28.784051 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:59:28.784070 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:59:28.784105 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:59:28.786400 (+ 2295us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:59:28.786974 (+ 574us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:59:28.786983 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:59:28.786991 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:59:28.787149 (+ 158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:59:28.787547 (+ 398us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:59:28.787551 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:59:28.787553 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:59:28.788124 (+ 571us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:59:28.788974 (+ 850us) server_negotiation.cc:1035] Waiting for connection context
0812 06:59:28.789153 (+ 179us) server_negotiation.cc:299] Negotiation successful
0812 06:59:28.789385 (+ 232us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":373,"thread_start_us":167,"threads_started":1}
I20260812 06:59:28.790426 11287 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5085ebf85cb446d7a5d8cff76edb90a1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5085ebf85cb446d7a5d8cff76edb90a1. Latest consensus state: current_term: 1 leader_uuid: "5085ebf85cb446d7a5d8cff76edb90a1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5085ebf85cb446d7a5d8cff76edb90a1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34239 } } }
I20260812 06:59:28.790567 11287 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5085ebf85cb446d7a5d8cff76edb90a1 [sys.catalog]: This master's current role is: LEADER
I20260812 06:59:28.790900 11285 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5085ebf85cb446d7a5d8cff76edb90a1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5085ebf85cb446d7a5d8cff76edb90a1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5085ebf85cb446d7a5d8cff76edb90a1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34239 } } }
I20260812 06:59:28.791029 11285 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5085ebf85cb446d7a5d8cff76edb90a1 [sys.catalog]: This master's current role is: LEADER
I20260812 06:59:28.791486 11294 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 06:59:28.794236 19665 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 06:59:28.795204 11294 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 06:59:28.802186 11294 catalog_manager.cc:1347] Generated new cluster ID: 5f486cac8bf34a0293c5ae41d7ce7714
I20260812 06:59:28.802274 11294 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 06:59:28.807142 11282 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:59:28.796781 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.40:55459 (local address 127.0.0.1:34239)
0812 06:59:28.796947 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0812 06:59:28.796953 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 06:59:28.797656 (+ 703us) server_negotiation.cc:407] Connection header received
0812 06:59:28.797789 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:59:28.797792 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:59:28.797861 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:59:28.797972 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:59:28.798903 (+ 931us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:59:28.799831 (+ 928us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:59:28.800876 (+ 1045us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:59:28.801140 (+ 264us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:59:28.801264 (+ 124us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:59:28.803143 (+ 1879us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:59:28.803147 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:59:28.803150 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:59:28.803182 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:59:28.805081 (+ 1899us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:59:28.805695 (+ 614us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:59:28.805699 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:59:28.805701 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:59:28.805804 (+ 103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:59:28.806167 (+ 363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:59:28.806171 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:59:28.806173 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:59:28.806415 (+ 242us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:59:28.806568 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0812 06:59:28.806853 (+ 285us) server_negotiation.cc:299] Negotiation successful
0812 06:59:28.806964 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260812 06:59:28.824824 11294 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 06:59:28.826164 11294 catalog_manager.cc:1504] Loading token signing keys...
I20260812 06:59:28.836503 11294 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5085ebf85cb446d7a5d8cff76edb90a1: Generated new TSK 0
I20260812 06:59:28.837337 11294 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 06:59:28.845649 19665 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu with pid 11194
W20260812 06:59:28.854707 11179 connection.cc:535] client connection to 127.0.0.1:34239 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:34239): Connection reset by peer (error 104)
2026-08-12T06:59:28Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/6 (1603 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/7
Loading random data
Initializing database '/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786517687384214-19665-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 06:59:28 dist-test-slave-65mx krb5kdc[11312](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 06:59:28 dist-test-slave-65mx krb5kdc[11312](info): set up 2 sockets
Aug 12 06:59:28 dist-test-slave-65mx krb5kdc[11312](info): commencing operation
krb5kdc: starting...
W20260812 06:59:30.006832 19665 mini_kdc.cc:121] Time spent starting KDC: real 1.137s user 0.006s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786517687384214-19665-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786517687384214-19665-0/krb5kdc/test-user.keytab.
Aug 12 06:59:30 dist-test-slave-65mx krb5kdc[11312](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517970, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T06:59:30Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T06:59:30Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786517687384214-19665-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786517687384214-19665-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260812 06:59:30.148178 19665 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu
/tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786517687384214-19665-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786517687384214-19665-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786517687384214-19665-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786517687384214-19665-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38809
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786517687384214-19665-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:40737
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:38809
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786517687384214-19665-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786517687384214-19665-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786517687384214-19665-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 06:59:30.264045 11326 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:59:30.264389 11326 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:59:30.264475 11326 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 06:59:30.268658 11326 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 06:59:30.268771 11326 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 06:59:30.268807 11326 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 06:59:30.268849 11326 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 06:59:30.268884 11326 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 06:59:30.274461 11326 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:40737
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786517687384214-19665-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786517687384214-19665-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:38809
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786517687384214-19665-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38809
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786517687384214-19665-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11326
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786517687384214-19665-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 06:15:54 UTC on ccycloud.quasar-rnfzce.root.comops.site
build id 4581989
I20260812 06:59:30.275709 11326 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:59:30.276842 11326 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 06:59:30.285526 11333 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:59:30.285552 11337 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:59:30.285518 11334 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:59:30.285739 11326 server_base.cc:1034] running on GCE node
I20260812 06:59:30.286454 11326 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:59:30.287670 11326 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:59:30.288843 11326 hybrid_clock.cc:648] HybridClock initialized: now 1786517970288822 us; error 33 us; skew 500 ppm
Aug 12 06:59:30 dist-test-slave-65mx krb5kdc[11312](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517970, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 06:59:30.295920 11326 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260812 06:59:30.297060 11326 webserver.cc:460] Webserver started at http://127.0.0.1:45831/ using document root <none> and password file <none>
I20260812 06:59:30.297623 11326 fs_manager.cc:359] Metadata directory not provided
I20260812 06:59:30.297685 11326 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:59:30.297865 11326 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 06:59:30.299978 11326 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786517687384214-19665-0/minicluster-data/master-0/data/instance:
uuid: "816e37d1e80c4cc29788abde60d61375"
format_stamp: "Formatted at 2026-08-12 06:59:30 on dist-test-slave-65mx"
server_key: "be131d5a73478bd761b281c098fe5b39"
server_key_iv: "41b08bdd1aa9f13cfe8cf131be4f014f"
server_key_version: "encryptionkey@0"
I20260812 06:59:30.300587 11326 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786517687384214-19665-0/minicluster-data/master-0/wal/instance:
uuid: "816e37d1e80c4cc29788abde60d61375"
format_stamp: "Formatted at 2026-08-12 06:59:30 on dist-test-slave-65mx"
server_key: "be131d5a73478bd761b281c098fe5b39"
server_key_iv: "41b08bdd1aa9f13cfe8cf131be4f014f"
server_key_version: "encryptionkey@0"
I20260812 06:59:30.304628 11326 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260812 06:59:30.307274 11344 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 06:59:30.308763 11326 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 06:59:30.308913 11326 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786517687384214-19665-0/minicluster-data/master-0/data,/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786517687384214-19665-0/minicluster-data/master-0/wal
uuid: "816e37d1e80c4cc29788abde60d61375"
format_stamp: "Formatted at 2026-08-12 06:59:30 on dist-test-slave-65mx"
server_key: "be131d5a73478bd761b281c098fe5b39"
server_key_iv: "41b08bdd1aa9f13cfe8cf131be4f014f"
server_key_version: "encryptionkey@0"
I20260812 06:59:30.309048 11326 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786517687384214-19665-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786517687384214-19665-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786517687384214-19665-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:59:30.340286 11326 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 06:59:30.344017 11326 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 06:59:30.344254 11326 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:59:30.351969 11326 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:38809
I20260812 06:59:30.352015 11403 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:38809 every 8 connection(s)
I20260812 06:59:30.353070 11326 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786517687384214-19665-0/minicluster-data/master-0/data/info.pb
I20260812 06:59:30.356035 11404 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 06:59:30.356853 19665 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu as pid 11326
I20260812 06:59:30.357020 19665 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786517687384214-19665-0/minicluster-data/master-0/wal/instance
I20260812 06:59:30.357312 19665 external_mini_cluster.cc:1428] Setting key 94393770596da1fd4b98abeab2d47113
I20260812 06:59:30.362356 11404 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 816e37d1e80c4cc29788abde60d61375: Bootstrap starting.
I20260812 06:59:30.364715 11404 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 816e37d1e80c4cc29788abde60d61375: Neither blocks nor log segments found. Creating new log.
I20260812 06:59:30.365455 11404 log.cc:824] T 00000000000000000000000000000000 P 816e37d1e80c4cc29788abde60d61375: Log is configured to *not* fsync() on all Append() calls
I20260812 06:59:30.366760 11408 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:59:30.358846 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.40:60167 (local address 127.0.0.1:38809)
0812 06:59:30.359299 (+ 453us) server_negotiation.cc:206] Beginning negotiation
0812 06:59:30.359314 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0812 06:59:30.359350 (+ 36us) server_negotiation.cc:407] Connection header received
0812 06:59:30.360005 (+ 655us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:59:30.360021 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:59:30.360363 (+ 342us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:59:30.360830 (+ 467us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:59:30.361252 (+ 422us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:59:30.362203 (+ 951us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:59:30.363158 (+ 955us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:59:30.363640 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:59:30.363809 (+ 169us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:59:30.364830 (+ 1021us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:59:30.364840 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:59:30.364858 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 06:59:30.365279 (+ 421us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:59:30.365333 (+ 54us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 06:59:30.365355 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:59:30.365500 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0812 06:59:30.365730 (+ 230us) server_negotiation.cc:299] Negotiation successful
0812 06:59:30.365882 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":291,"thread_start_us":148,"threads_started":1}
I20260812 06:59:30.367717 11404 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 816e37d1e80c4cc29788abde60d61375: No bootstrap required, opened a new log
I20260812 06:59:30.369867 19665 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 06:59:30.371893 11404 raft_consensus.cc:348] T 00000000000000000000000000000000 P 816e37d1e80c4cc29788abde60d61375 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "816e37d1e80c4cc29788abde60d61375" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38809 } }
I20260812 06:59:30.372222 11404 raft_consensus.cc:374] T 00000000000000000000000000000000 P 816e37d1e80c4cc29788abde60d61375 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 06:59:30.372309 11404 raft_consensus.cc:729] T 00000000000000000000000000000000 P 816e37d1e80c4cc29788abde60d61375 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 816e37d1e80c4cc29788abde60d61375, State: Initialized, Role: FOLLOWER
I20260812 06:59:30.372800 11404 consensus_queue.cc:260] T 00000000000000000000000000000000 P 816e37d1e80c4cc29788abde60d61375 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "816e37d1e80c4cc29788abde60d61375" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38809 } }
I20260812 06:59:30.372956 11404 raft_consensus.cc:388] T 00000000000000000000000000000000 P 816e37d1e80c4cc29788abde60d61375 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 06:59:30.373050 11404 raft_consensus.cc:482] T 00000000000000000000000000000000 P 816e37d1e80c4cc29788abde60d61375 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 06:59:30.373169 11404 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 816e37d1e80c4cc29788abde60d61375 [term 0 FOLLOWER]: Advancing to term 1
I20260812 06:59:30.374177 11404 raft_consensus.cc:504] T 00000000000000000000000000000000 P 816e37d1e80c4cc29788abde60d61375 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "816e37d1e80c4cc29788abde60d61375" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38809 } }
I20260812 06:59:30.374573 11404 leader_election.cc:302] T 00000000000000000000000000000000 P 816e37d1e80c4cc29788abde60d61375 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 816e37d1e80c4cc29788abde60d61375; no voters:
I20260812 06:59:30.374945 11404 leader_election.cc:288] T 00000000000000000000000000000000 P 816e37d1e80c4cc29788abde60d61375 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 06:59:30.375073 11416 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 816e37d1e80c4cc29788abde60d61375 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 06:59:30.375273 11416 raft_consensus.cc:686] T 00000000000000000000000000000000 P 816e37d1e80c4cc29788abde60d61375 [term 1 LEADER]: Becoming Leader. State: Replica: 816e37d1e80c4cc29788abde60d61375, State: Running, Role: LEADER
I20260812 06:59:30.375581 11416 consensus_queue.cc:237] T 00000000000000000000000000000000 P 816e37d1e80c4cc29788abde60d61375 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "816e37d1e80c4cc29788abde60d61375" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38809 } }
I20260812 06:59:30.376350 11404 sys_catalog.cc:564] T 00000000000000000000000000000000 P 816e37d1e80c4cc29788abde60d61375 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 06:59:30.376533 11408 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:59:30.371966 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.40:52255 (local address 127.0.0.1:38809)
0812 06:59:30.372095 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0812 06:59:30.372099 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 06:59:30.372327 (+ 228us) server_negotiation.cc:407] Connection header received
0812 06:59:30.372454 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:59:30.372457 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:59:30.372511 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:59:30.372637 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:59:30.372939 (+ 302us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:59:30.373615 (+ 676us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:59:30.374482 (+ 867us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:59:30.374795 (+ 313us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:59:30.374914 (+ 119us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:59:30.375959 (+ 1045us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:59:30.375963 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:59:30.375965 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 06:59:30.375979 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:59:30.376008 (+ 29us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 06:59:30.376020 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:59:30.376081 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0812 06:59:30.376281 (+ 200us) server_negotiation.cc:299] Negotiation successful
0812 06:59:30.376320 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260812 06:59:30.376816 11416 sys_catalog.cc:455] T 00000000000000000000000000000000 P 816e37d1e80c4cc29788abde60d61375 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 816e37d1e80c4cc29788abde60d61375. Latest consensus state: current_term: 1 leader_uuid: "816e37d1e80c4cc29788abde60d61375" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "816e37d1e80c4cc29788abde60d61375" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38809 } } }
I20260812 06:59:30.376948 11416 sys_catalog.cc:458] T 00000000000000000000000000000000 P 816e37d1e80c4cc29788abde60d61375 [sys.catalog]: This master's current role is: LEADER
I20260812 06:59:30.377681 11420 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 06:59:30.377658 11418 sys_catalog.cc:455] T 00000000000000000000000000000000 P 816e37d1e80c4cc29788abde60d61375 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "816e37d1e80c4cc29788abde60d61375" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "816e37d1e80c4cc29788abde60d61375" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38809 } } }
I20260812 06:59:30.377806 11418 sys_catalog.cc:458] T 00000000000000000000000000000000 P 816e37d1e80c4cc29788abde60d61375 [sys.catalog]: This master's current role is: LEADER
I20260812 06:59:30.381341 11420 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 06:59:30.388247 11420 catalog_manager.cc:1347] Generated new cluster ID: 77e294382e5743a0a5050934d7fdc2c0
I20260812 06:59:30.388355 11420 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 06:59:30.399152 11420 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 06:59:30.400175 11420 catalog_manager.cc:1504] Loading token signing keys...
I20260812 06:59:30.404247 11420 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 816e37d1e80c4cc29788abde60d61375: Generated new TSK 0
I20260812 06:59:30.405123 11420 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 06:59:30.418668 19665 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu with pid 11326
W20260812 06:59:30.428359 11310 connection.cc:535] client connection to 127.0.0.1:38809 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:38809): Connection reset by peer (error 104)
2026-08-12T06:59:30Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/7 (1573 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/8
Loading random data
Initializing database '/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786517687384214-19665-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 06:59:30 dist-test-slave-65mx krb5kdc[11439](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 06:59:30 dist-test-slave-65mx krb5kdc[11439](info): set up 2 sockets
Aug 12 06:59:30 dist-test-slave-65mx krb5kdc[11439](info): commencing operation
krb5kdc: starting...
W20260812 06:59:31.558139 19665 mini_kdc.cc:121] Time spent starting KDC: real 1.115s user 0.000s sys 0.009s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786517687384214-19665-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786517687384214-19665-0/krb5kdc/test-user.keytab.
Aug 12 06:59:31 dist-test-slave-65mx krb5kdc[11439](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517971, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T06:59:31Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T06:59:31Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786517687384214-19665-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786517687384214-19665-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260812 06:59:31.696152 19665 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu
/tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786517687384214-19665-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786517687384214-19665-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786517687384214-19665-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786517687384214-19665-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33705
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786517687384214-19665-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:37379
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:33705
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786517687384214-19665-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786517687384214-19665-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786517687384214-19665-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 06:59:31.812134 11456 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:59:31.812433 11456 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:59:31.812477 11456 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 06:59:31.816618 11456 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 06:59:31.816689 11456 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 06:59:31.816708 11456 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 06:59:31.816725 11456 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 06:59:31.816741 11456 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 06:59:31.822178 11456 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:37379
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786517687384214-19665-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786517687384214-19665-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:33705
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786517687384214-19665-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33705
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786517687384214-19665-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11456
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786517687384214-19665-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 06:15:54 UTC on ccycloud.quasar-rnfzce.root.comops.site
build id 4581989
I20260812 06:59:31.823352 11456 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:59:31.824416 11456 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 06:59:31.833442 11464 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:59:31.833586 11456 server_base.cc:1034] running on GCE node
W20260812 06:59:31.833616 11461 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:59:31.833464 11462 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:59:31.834375 11456 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:59:31.835589 11456 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:59:31.836786 11456 hybrid_clock.cc:648] HybridClock initialized: now 1786517971836764 us; error 43 us; skew 500 ppm
Aug 12 06:59:31 dist-test-slave-65mx krb5kdc[11439](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517971, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 06:59:31.840051 11456 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260812 06:59:31.841086 11456 webserver.cc:460] Webserver started at http://127.0.0.1:38299/ using document root <none> and password file <none>
I20260812 06:59:31.841643 11456 fs_manager.cc:359] Metadata directory not provided
I20260812 06:59:31.841706 11456 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:59:31.841879 11456 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 06:59:31.843937 11456 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786517687384214-19665-0/minicluster-data/master-0/data/instance:
uuid: "4ff4c54da26f4f8aa16605021ed221b3"
format_stamp: "Formatted at 2026-08-12 06:59:31 on dist-test-slave-65mx"
server_key: "4ab84dfec4b4c2f6862c76c2fd1bfed3"
server_key_iv: "ee0c0977b8b6b68bfa11339c9cbe6224"
server_key_version: "encryptionkey@0"
I20260812 06:59:31.844776 11456 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786517687384214-19665-0/minicluster-data/master-0/wal/instance:
uuid: "4ff4c54da26f4f8aa16605021ed221b3"
format_stamp: "Formatted at 2026-08-12 06:59:31 on dist-test-slave-65mx"
server_key: "4ab84dfec4b4c2f6862c76c2fd1bfed3"
server_key_iv: "ee0c0977b8b6b68bfa11339c9cbe6224"
server_key_version: "encryptionkey@0"
I20260812 06:59:31.848817 11456 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260812 06:59:31.851512 11471 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 06:59:31.853065 11456 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 06:59:31.853174 11456 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786517687384214-19665-0/minicluster-data/master-0/data,/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786517687384214-19665-0/minicluster-data/master-0/wal
uuid: "4ff4c54da26f4f8aa16605021ed221b3"
format_stamp: "Formatted at 2026-08-12 06:59:31 on dist-test-slave-65mx"
server_key: "4ab84dfec4b4c2f6862c76c2fd1bfed3"
server_key_iv: "ee0c0977b8b6b68bfa11339c9cbe6224"
server_key_version: "encryptionkey@0"
I20260812 06:59:31.853277 11456 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786517687384214-19665-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786517687384214-19665-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786517687384214-19665-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:59:31.857904 11456 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 06:59:31.861352 11456 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 06:59:31.861533 11456 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:59:31.869102 11456 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:33705
I20260812 06:59:31.869130 11528 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:33705 every 8 connection(s)
I20260812 06:59:31.870141 11456 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786517687384214-19665-0/minicluster-data/master-0/data/info.pb
I20260812 06:59:31.872988 11529 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 06:59:31.874276 19665 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu as pid 11456
I20260812 06:59:31.874434 19665 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786517687384214-19665-0/minicluster-data/master-0/wal/instance
I20260812 06:59:31.874744 19665 external_mini_cluster.cc:1428] Setting key 609267d4ee9ee8dcac065ce8d731d4f9
I20260812 06:59:31.879343 11529 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4ff4c54da26f4f8aa16605021ed221b3: Bootstrap starting.
I20260812 06:59:31.881637 11529 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4ff4c54da26f4f8aa16605021ed221b3: Neither blocks nor log segments found. Creating new log.
I20260812 06:59:31.882308 11529 log.cc:824] T 00000000000000000000000000000000 P 4ff4c54da26f4f8aa16605021ed221b3: Log is configured to *not* fsync() on all Append() calls
I20260812 06:59:31.884833 11529 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4ff4c54da26f4f8aa16605021ed221b3: No bootstrap required, opened a new log
I20260812 06:59:31.884888 11532 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:59:31.876310 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.40:56197 (local address 127.0.0.1:33705)
0812 06:59:31.876782 (+ 472us) server_negotiation.cc:206] Beginning negotiation
0812 06:59:31.876797 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0812 06:59:31.876833 (+ 36us) server_negotiation.cc:407] Connection header received
0812 06:59:31.877513 (+ 680us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:59:31.877530 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:59:31.877820 (+ 290us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:59:31.878164 (+ 344us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:59:31.883363 (+ 5199us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:59:31.883372 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:59:31.883392 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 06:59:31.883407 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:59:31.883501 (+ 94us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 06:59:31.883533 (+ 32us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:59:31.883661 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0812 06:59:31.883789 (+ 128us) server_negotiation.cc:299] Negotiation successful
0812 06:59:31.883947 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":323,"thread_start_us":151,"threads_started":1}
I20260812 06:59:31.887713 19665 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 06:59:31.888861 11529 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4ff4c54da26f4f8aa16605021ed221b3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4ff4c54da26f4f8aa16605021ed221b3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33705 } }
I20260812 06:59:31.889133 11529 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4ff4c54da26f4f8aa16605021ed221b3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 06:59:31.889230 11529 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4ff4c54da26f4f8aa16605021ed221b3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4ff4c54da26f4f8aa16605021ed221b3, State: Initialized, Role: FOLLOWER
I20260812 06:59:31.889714 11529 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4ff4c54da26f4f8aa16605021ed221b3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4ff4c54da26f4f8aa16605021ed221b3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33705 } }
I20260812 06:59:31.889842 11529 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4ff4c54da26f4f8aa16605021ed221b3 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 06:59:31.889951 11529 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4ff4c54da26f4f8aa16605021ed221b3 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 06:59:31.890096 11529 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4ff4c54da26f4f8aa16605021ed221b3 [term 0 FOLLOWER]: Advancing to term 1
I20260812 06:59:31.891443 11529 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4ff4c54da26f4f8aa16605021ed221b3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4ff4c54da26f4f8aa16605021ed221b3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33705 } }
I20260812 06:59:31.891850 11529 leader_election.cc:302] T 00000000000000000000000000000000 P 4ff4c54da26f4f8aa16605021ed221b3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4ff4c54da26f4f8aa16605021ed221b3; no voters:
I20260812 06:59:31.892283 11529 leader_election.cc:288] T 00000000000000000000000000000000 P 4ff4c54da26f4f8aa16605021ed221b3 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 06:59:31.892378 11539 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4ff4c54da26f4f8aa16605021ed221b3 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 06:59:31.892493 11532 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:59:31.890611 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.40:44257 (local address 127.0.0.1:33705)
0812 06:59:31.890776 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0812 06:59:31.890781 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 06:59:31.890913 (+ 132us) server_negotiation.cc:407] Connection header received
0812 06:59:31.891052 (+ 139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:59:31.891054 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:59:31.891086 (+ 32us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:59:31.891154 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:59:31.891973 (+ 819us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:59:31.891976 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:59:31.891978 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 06:59:31.892007 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:59:31.892041 (+ 34us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 06:59:31.892061 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:59:31.892135 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0812 06:59:31.892265 (+ 130us) server_negotiation.cc:299] Negotiation successful
0812 06:59:31.892309 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260812 06:59:31.892729 11539 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4ff4c54da26f4f8aa16605021ed221b3 [term 1 LEADER]: Becoming Leader. State: Replica: 4ff4c54da26f4f8aa16605021ed221b3, State: Running, Role: LEADER
I20260812 06:59:31.893059 11539 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4ff4c54da26f4f8aa16605021ed221b3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4ff4c54da26f4f8aa16605021ed221b3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33705 } }
I20260812 06:59:31.893730 11529 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4ff4c54da26f4f8aa16605021ed221b3 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 06:59:31.894371 11539 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4ff4c54da26f4f8aa16605021ed221b3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4ff4c54da26f4f8aa16605021ed221b3. Latest consensus state: current_term: 1 leader_uuid: "4ff4c54da26f4f8aa16605021ed221b3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4ff4c54da26f4f8aa16605021ed221b3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33705 } } }
I20260812 06:59:31.894368 11542 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4ff4c54da26f4f8aa16605021ed221b3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4ff4c54da26f4f8aa16605021ed221b3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4ff4c54da26f4f8aa16605021ed221b3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33705 } } }
I20260812 06:59:31.894598 11542 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4ff4c54da26f4f8aa16605021ed221b3 [sys.catalog]: This master's current role is: LEADER
I20260812 06:59:31.894865 11539 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4ff4c54da26f4f8aa16605021ed221b3 [sys.catalog]: This master's current role is: LEADER
I20260812 06:59:31.896230 11549 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 06:59:31.899266 11549 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 06:59:31.906078 11549 catalog_manager.cc:1347] Generated new cluster ID: 24268544ed9f44529de61e4b0c0af0df
I20260812 06:59:31.906170 11549 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 06:59:31.913976 11549 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 06:59:31.914328 11549 catalog_manager.cc:1504] Loading token signing keys...
I20260812 06:59:31.923017 11549 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4ff4c54da26f4f8aa16605021ed221b3: Generated new TSK 0
I20260812 06:59:31.923869 11549 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 06:59:31.929215 19665 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu with pid 11456
2026-08-12T06:59:31Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/8 (1513 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/9
Loading random data
Initializing database '/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786517687384214-19665-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 06:59:31 dist-test-slave-65mx krb5kdc[11562](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 06:59:31 dist-test-slave-65mx krb5kdc[11562](info): set up 2 sockets
Aug 12 06:59:31 dist-test-slave-65mx krb5kdc[11562](info): commencing operation
krb5kdc: starting...
W20260812 06:59:33.101989 19665 mini_kdc.cc:121] Time spent starting KDC: real 1.145s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786517687384214-19665-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786517687384214-19665-0/krb5kdc/test-user.keytab.
Aug 12 06:59:33 dist-test-slave-65mx krb5kdc[11562](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517973, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T06:59:33Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T06:59:33Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786517687384214-19665-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786517687384214-19665-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260812 06:59:33.330281 19665 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu
/tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786517687384214-19665-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786517687384214-19665-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786517687384214-19665-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786517687384214-19665-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46173
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786517687384214-19665-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:39831
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:46173
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786517687384214-19665-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786517687384214-19665-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786517687384214-19665-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 06:59:33.585892 11579 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:59:33.586404 11579 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:59:33.586565 11579 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 06:59:33.593403 11579 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 06:59:33.593525 11579 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 06:59:33.593560 11579 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 06:59:33.593637 11579 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 06:59:33.593704 11579 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 06:59:33.602659 11579 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:39831
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786517687384214-19665-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786517687384214-19665-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:46173
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786517687384214-19665-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46173
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786517687384214-19665-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11579
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786517687384214-19665-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 06:15:54 UTC on ccycloud.quasar-rnfzce.root.comops.site
build id 4581989
I20260812 06:59:33.604592 11579 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:59:33.606390 11579 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 06:59:33.621865 11584 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:59:33.622032 11585 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:59:33.622032 11588 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:59:33.623842 11579 server_base.cc:1034] running on GCE node
I20260812 06:59:33.624950 11579 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:59:33.626920 11579 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:59:33.628198 11579 hybrid_clock.cc:648] HybridClock initialized: now 1786517973628123 us; error 86 us; skew 500 ppm
Aug 12 06:59:33 dist-test-slave-65mx krb5kdc[11562](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517973, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 06:59:33.634693 11579 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260812 06:59:33.636664 11579 webserver.cc:460] Webserver started at http://127.0.0.1:45293/ using document root <none> and password file <none>
I20260812 06:59:33.637753 11579 fs_manager.cc:359] Metadata directory not provided
I20260812 06:59:33.637917 11579 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:59:33.638574 11579 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 06:59:33.642740 11579 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786517687384214-19665-0/minicluster-data/master-0/data/instance:
uuid: "46455562412f43ca8ed6e520a4f7aa81"
format_stamp: "Formatted at 2026-08-12 06:59:33 on dist-test-slave-65mx"
server_key: "37ad1d6238c627074a3f886840f28946"
server_key_iv: "afa00587168fa0035adace2a031a7000"
server_key_version: "encryptionkey@0"
I20260812 06:59:33.644250 11579 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786517687384214-19665-0/minicluster-data/master-0/wal/instance:
uuid: "46455562412f43ca8ed6e520a4f7aa81"
format_stamp: "Formatted at 2026-08-12 06:59:33 on dist-test-slave-65mx"
server_key: "37ad1d6238c627074a3f886840f28946"
server_key_iv: "afa00587168fa0035adace2a031a7000"
server_key_version: "encryptionkey@0"
I20260812 06:59:33.653407 11579 fs_manager.cc:691] Time spent creating directory manager: real 0.009s user 0.007s sys 0.000s
I20260812 06:59:33.658844 11597 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 06:59:33.661530 11579 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.003s sys 0.000s
I20260812 06:59:33.661818 11579 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786517687384214-19665-0/minicluster-data/master-0/data,/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786517687384214-19665-0/minicluster-data/master-0/wal
uuid: "46455562412f43ca8ed6e520a4f7aa81"
format_stamp: "Formatted at 2026-08-12 06:59:33 on dist-test-slave-65mx"
server_key: "37ad1d6238c627074a3f886840f28946"
server_key_iv: "afa00587168fa0035adace2a031a7000"
server_key_version: "encryptionkey@0"
I20260812 06:59:33.662050 11579 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786517687384214-19665-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786517687384214-19665-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786517687384214-19665-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:59:33.693583 11579 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 06:59:33.700335 11579 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 06:59:33.700850 11579 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:59:33.716048 11579 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:46173
I20260812 06:59:33.716167 11655 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:46173 every 8 connection(s)
I20260812 06:59:33.717983 11579 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786517687384214-19665-0/minicluster-data/master-0/data/info.pb
I20260812 06:59:33.719537 19665 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu as pid 11579
I20260812 06:59:33.719801 19665 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786517687384214-19665-0/minicluster-data/master-0/wal/instance
I20260812 06:59:33.720269 19665 external_mini_cluster.cc:1428] Setting key 1d87374812ec0d2d6015a2426ad8a36c
I20260812 06:59:33.724421 11656 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 06:59:33.734915 11656 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 46455562412f43ca8ed6e520a4f7aa81: Bootstrap starting.
Aug 12 06:59:33 dist-test-slave-65mx krb5kdc[11562](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517973, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260812 06:59:33.740640 11656 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 46455562412f43ca8ed6e520a4f7aa81: Neither blocks nor log segments found. Creating new log.
I20260812 06:59:33.742170 11656 log.cc:824] T 00000000000000000000000000000000 P 46455562412f43ca8ed6e520a4f7aa81: Log is configured to *not* fsync() on all Append() calls
I20260812 06:59:33.747274 11656 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 46455562412f43ca8ed6e520a4f7aa81: No bootstrap required, opened a new log
I20260812 06:59:33.749018 11659 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:59:33.723090 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.40:41201 (local address 127.0.0.1:46173)
0812 06:59:33.723955 (+ 865us) server_negotiation.cc:206] Beginning negotiation
0812 06:59:33.723983 (+ 28us) server_negotiation.cc:399] Waiting for connection header
0812 06:59:33.724051 (+ 68us) server_negotiation.cc:407] Connection header received
0812 06:59:33.725173 (+ 1122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:59:33.725212 (+ 39us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:59:33.725793 (+ 581us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:59:33.726423 (+ 630us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:59:33.727932 (+ 1509us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:59:33.729926 (+ 1994us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:59:33.731519 (+ 1593us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:59:33.732400 (+ 881us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:59:33.732807 (+ 407us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:59:33.738589 (+ 5782us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:59:33.738598 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:59:33.738622 (+ 24us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:59:33.738694 (+ 72us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:59:33.742162 (+ 3468us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:59:33.743575 (+ 1413us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:59:33.743594 (+ 19us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:59:33.743601 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:59:33.744049 (+ 448us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:59:33.745020 (+ 971us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:59:33.745023 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:59:33.745024 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:59:33.745815 (+ 791us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:59:33.746078 (+ 263us) server_negotiation.cc:1035] Waiting for connection context
0812 06:59:33.746836 (+ 758us) server_negotiation.cc:299] Negotiation successful
0812 06:59:33.747344 (+ 508us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":605,"thread_start_us":364,"threads_started":1}
I20260812 06:59:33.754904 19665 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 06:59:33.754652 11656 raft_consensus.cc:348] T 00000000000000000000000000000000 P 46455562412f43ca8ed6e520a4f7aa81 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "46455562412f43ca8ed6e520a4f7aa81" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46173 } }
I20260812 06:59:33.755162 11656 raft_consensus.cc:374] T 00000000000000000000000000000000 P 46455562412f43ca8ed6e520a4f7aa81 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 06:59:33.755262 11656 raft_consensus.cc:729] T 00000000000000000000000000000000 P 46455562412f43ca8ed6e520a4f7aa81 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 46455562412f43ca8ed6e520a4f7aa81, State: Initialized, Role: FOLLOWER
I20260812 06:59:33.756178 11656 consensus_queue.cc:260] T 00000000000000000000000000000000 P 46455562412f43ca8ed6e520a4f7aa81 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "46455562412f43ca8ed6e520a4f7aa81" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46173 } }
I20260812 06:59:33.756493 11656 raft_consensus.cc:388] T 00000000000000000000000000000000 P 46455562412f43ca8ed6e520a4f7aa81 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 06:59:33.756644 11656 raft_consensus.cc:482] T 00000000000000000000000000000000 P 46455562412f43ca8ed6e520a4f7aa81 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 06:59:33.756845 11656 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 46455562412f43ca8ed6e520a4f7aa81 [term 0 FOLLOWER]: Advancing to term 1
I20260812 06:59:33.759066 11656 raft_consensus.cc:504] T 00000000000000000000000000000000 P 46455562412f43ca8ed6e520a4f7aa81 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "46455562412f43ca8ed6e520a4f7aa81" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46173 } }
I20260812 06:59:33.759783 11656 leader_election.cc:302] T 00000000000000000000000000000000 P 46455562412f43ca8ed6e520a4f7aa81 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 46455562412f43ca8ed6e520a4f7aa81; no voters:
I20260812 06:59:33.760383 11656 leader_election.cc:288] T 00000000000000000000000000000000 P 46455562412f43ca8ed6e520a4f7aa81 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 06:59:33.760650 11668 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 46455562412f43ca8ed6e520a4f7aa81 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 06:59:33.761143 11668 raft_consensus.cc:686] T 00000000000000000000000000000000 P 46455562412f43ca8ed6e520a4f7aa81 [term 1 LEADER]: Becoming Leader. State: Replica: 46455562412f43ca8ed6e520a4f7aa81, State: Running, Role: LEADER
I20260812 06:59:33.761876 11668 consensus_queue.cc:237] T 00000000000000000000000000000000 P 46455562412f43ca8ed6e520a4f7aa81 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "46455562412f43ca8ed6e520a4f7aa81" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46173 } }
I20260812 06:59:33.762872 11656 sys_catalog.cc:564] T 00000000000000000000000000000000 P 46455562412f43ca8ed6e520a4f7aa81 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 06:59:33.764153 11668 sys_catalog.cc:455] T 00000000000000000000000000000000 P 46455562412f43ca8ed6e520a4f7aa81 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 46455562412f43ca8ed6e520a4f7aa81. Latest consensus state: current_term: 1 leader_uuid: "46455562412f43ca8ed6e520a4f7aa81" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "46455562412f43ca8ed6e520a4f7aa81" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46173 } } }
I20260812 06:59:33.764379 11668 sys_catalog.cc:458] T 00000000000000000000000000000000 P 46455562412f43ca8ed6e520a4f7aa81 [sys.catalog]: This master's current role is: LEADER
I20260812 06:59:33.765203 11675 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 06:59:33.765197 11670 sys_catalog.cc:455] T 00000000000000000000000000000000 P 46455562412f43ca8ed6e520a4f7aa81 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "46455562412f43ca8ed6e520a4f7aa81" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "46455562412f43ca8ed6e520a4f7aa81" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46173 } } }
I20260812 06:59:33.765437 11670 sys_catalog.cc:458] T 00000000000000000000000000000000 P 46455562412f43ca8ed6e520a4f7aa81 [sys.catalog]: This master's current role is: LEADER
I20260812 06:59:33.772580 11675 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 06:59:33.778906 11659 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:59:33.758962 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.40:57727 (local address 127.0.0.1:46173)
0812 06:59:33.759204 (+ 242us) server_negotiation.cc:206] Beginning negotiation
0812 06:59:33.759223 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0812 06:59:33.759550 (+ 327us) server_negotiation.cc:407] Connection header received
0812 06:59:33.759714 (+ 164us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:59:33.759719 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:59:33.759862 (+ 143us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:59:33.760045 (+ 183us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:59:33.761973 (+ 1928us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:59:33.763127 (+ 1154us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:59:33.765389 (+ 2262us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:59:33.766065 (+ 676us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:59:33.766261 (+ 196us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:59:33.770970 (+ 4709us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:59:33.770976 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:59:33.770992 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:59:33.771039 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:59:33.774471 (+ 3432us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:59:33.775841 (+ 1370us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:59:33.775856 (+ 15us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:59:33.775859 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:59:33.776035 (+ 176us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:59:33.776993 (+ 958us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:59:33.776997 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:59:33.776999 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:59:33.777458 (+ 459us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:59:33.777740 (+ 282us) server_negotiation.cc:1035] Waiting for connection context
0812 06:59:33.778376 (+ 636us) server_negotiation.cc:299] Negotiation successful
0812 06:59:33.778603 (+ 227us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":84}
I20260812 06:59:33.785382 11675 catalog_manager.cc:1347] Generated new cluster ID: 3772f3a40f8742f1b29ad4e6da453499
I20260812 06:59:33.785627 11675 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 06:59:33.813880 11675 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 06:59:33.815923 11675 catalog_manager.cc:1504] Loading token signing keys...
I20260812 06:59:33.831643 11675 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 46455562412f43ca8ed6e520a4f7aa81: Generated new TSK 0
I20260812 06:59:33.833057 11675 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 06:59:33.842254 19665 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu with pid 11579
W20260812 06:59:33.856062 11560 connection.cc:535] client connection to 127.0.0.1:46173 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:46173): Connection reset by peer (error 104)
2026-08-12T06:59:33Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/9 (1916 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/10
Loading random data
Initializing database '/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786517687384214-19665-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 06:59:33 dist-test-slave-65mx krb5kdc[11693](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 06:59:33 dist-test-slave-65mx krb5kdc[11693](info): set up 2 sockets
Aug 12 06:59:33 dist-test-slave-65mx krb5kdc[11693](info): commencing operation
krb5kdc: starting...
W20260812 06:59:35.554088 19665 mini_kdc.cc:121] Time spent starting KDC: real 1.680s user 0.000s sys 0.011s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786517687384214-19665-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786517687384214-19665-0/krb5kdc/test-user.keytab.
Aug 12 06:59:35 dist-test-slave-65mx krb5kdc[11693](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517975, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T06:59:35Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T06:59:35Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786517687384214-19665-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786517687384214-19665-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260812 06:59:35.700143 19665 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu
/tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786517687384214-19665-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786517687384214-19665-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786517687384214-19665-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786517687384214-19665-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44407
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786517687384214-19665-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:46175
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:44407
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786517687384214-19665-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786517687384214-19665-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786517687384214-19665-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 06:59:35.822328 11708 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:59:35.822702 11708 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:59:35.822791 11708 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 06:59:35.827281 11708 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 06:59:35.827409 11708 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 06:59:35.827446 11708 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 06:59:35.827490 11708 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 06:59:35.827550 11708 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 06:59:35.833370 11708 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:46175
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786517687384214-19665-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786517687384214-19665-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:44407
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786517687384214-19665-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44407
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786517687384214-19665-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11708
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786517687384214-19665-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 06:15:54 UTC on ccycloud.quasar-rnfzce.root.comops.site
build id 4581989
I20260812 06:59:35.834627 11708 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:59:35.835779 11708 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 06:59:35.845674 11714 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:59:35.845690 11716 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:59:35.845718 11713 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:59:35.845947 11708 server_base.cc:1034] running on GCE node
I20260812 06:59:35.846699 11708 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:59:35.847960 11708 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:59:35.849180 11708 hybrid_clock.cc:648] HybridClock initialized: now 1786517975849162 us; error 33 us; skew 500 ppm
Aug 12 06:59:35 dist-test-slave-65mx krb5kdc[11693](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517975, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 06:59:35.856462 11708 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260812 06:59:35.858392 11708 webserver.cc:460] Webserver started at http://127.0.0.1:37195/ using document root <none> and password file <none>
I20260812 06:59:35.859218 11708 fs_manager.cc:359] Metadata directory not provided
I20260812 06:59:35.859356 11708 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:59:35.859660 11708 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 06:59:35.862371 11708 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786517687384214-19665-0/minicluster-data/master-0/data/instance:
uuid: "5e9b1f8f888e4119a7d737540089c507"
format_stamp: "Formatted at 2026-08-12 06:59:35 on dist-test-slave-65mx"
server_key: "2be39375ffeae899af8f4698078c23e7"
server_key_iv: "10ca95f2dc82f77d99fdb7fe8d84951f"
server_key_version: "encryptionkey@0"
I20260812 06:59:35.863092 11708 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786517687384214-19665-0/minicluster-data/master-0/wal/instance:
uuid: "5e9b1f8f888e4119a7d737540089c507"
format_stamp: "Formatted at 2026-08-12 06:59:35 on dist-test-slave-65mx"
server_key: "2be39375ffeae899af8f4698078c23e7"
server_key_iv: "10ca95f2dc82f77d99fdb7fe8d84951f"
server_key_version: "encryptionkey@0"
I20260812 06:59:35.867889 11708 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260812 06:59:35.870826 11724 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 06:59:35.872306 11708 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260812 06:59:35.872491 11708 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786517687384214-19665-0/minicluster-data/master-0/data,/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786517687384214-19665-0/minicluster-data/master-0/wal
uuid: "5e9b1f8f888e4119a7d737540089c507"
format_stamp: "Formatted at 2026-08-12 06:59:35 on dist-test-slave-65mx"
server_key: "2be39375ffeae899af8f4698078c23e7"
server_key_iv: "10ca95f2dc82f77d99fdb7fe8d84951f"
server_key_version: "encryptionkey@0"
I20260812 06:59:35.872692 11708 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786517687384214-19665-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786517687384214-19665-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786517687384214-19665-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:59:35.907394 11708 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 06:59:35.911232 11708 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 06:59:35.911491 11708 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:59:35.919763 11708 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:44407
I20260812 06:59:35.919793 11782 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:44407 every 8 connection(s)
I20260812 06:59:35.920858 11708 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786517687384214-19665-0/minicluster-data/master-0/data/info.pb
I20260812 06:59:35.923763 11783 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 06:59:35.929749 19665 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu as pid 11708
I20260812 06:59:35.929877 11783 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5e9b1f8f888e4119a7d737540089c507: Bootstrap starting.
I20260812 06:59:35.929936 19665 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786517687384214-19665-0/minicluster-data/master-0/wal/instance
I20260812 06:59:35.930327 19665 external_mini_cluster.cc:1428] Setting key 01c9b95fd5c0c2b385a56cb22da609cd
I20260812 06:59:35.932667 11783 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5e9b1f8f888e4119a7d737540089c507: Neither blocks nor log segments found. Creating new log.
I20260812 06:59:35.933579 11783 log.cc:824] T 00000000000000000000000000000000 P 5e9b1f8f888e4119a7d737540089c507: Log is configured to *not* fsync() on all Append() calls
I20260812 06:59:35.936326 11783 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5e9b1f8f888e4119a7d737540089c507: No bootstrap required, opened a new log
I20260812 06:59:35.941236 11783 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5e9b1f8f888e4119a7d737540089c507 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5e9b1f8f888e4119a7d737540089c507" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44407 } }
I20260812 06:59:35.941529 11783 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5e9b1f8f888e4119a7d737540089c507 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 06:59:35.941556 11786 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:59:35.931791 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.40:50053 (local address 127.0.0.1:44407)
0812 06:59:35.932259 (+ 468us) server_negotiation.cc:206] Beginning negotiation
0812 06:59:35.932279 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0812 06:59:35.932322 (+ 43us) server_negotiation.cc:407] Connection header received
0812 06:59:35.933278 (+ 956us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:59:35.933301 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:59:35.933764 (+ 463us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:59:35.934130 (+ 366us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:59:35.934737 (+ 607us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:59:35.936180 (+ 1443us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:59:35.937312 (+ 1132us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:59:35.938019 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:59:35.938196 (+ 177us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:59:35.939216 (+ 1020us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:59:35.939222 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:59:35.939241 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 06:59:35.939885 (+ 644us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:59:35.939956 (+ 71us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 06:59:35.939987 (+ 31us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:59:35.940131 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0812 06:59:35.940339 (+ 208us) server_negotiation.cc:299] Negotiation successful
0812 06:59:35.940489 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":314,"thread_start_us":136,"threads_started":1}
I20260812 06:59:35.941592 11783 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5e9b1f8f888e4119a7d737540089c507 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5e9b1f8f888e4119a7d737540089c507, State: Initialized, Role: FOLLOWER
I20260812 06:59:35.942083 11783 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5e9b1f8f888e4119a7d737540089c507 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5e9b1f8f888e4119a7d737540089c507" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44407 } }
I20260812 06:59:35.942245 11783 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5e9b1f8f888e4119a7d737540089c507 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 06:59:35.942335 11783 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5e9b1f8f888e4119a7d737540089c507 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 06:59:35.942446 11783 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5e9b1f8f888e4119a7d737540089c507 [term 0 FOLLOWER]: Advancing to term 1
I20260812 06:59:35.943742 11783 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5e9b1f8f888e4119a7d737540089c507 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5e9b1f8f888e4119a7d737540089c507" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44407 } }
I20260812 06:59:35.944103 11783 leader_election.cc:302] T 00000000000000000000000000000000 P 5e9b1f8f888e4119a7d737540089c507 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5e9b1f8f888e4119a7d737540089c507; no voters:
I20260812 06:59:35.944453 11783 leader_election.cc:288] T 00000000000000000000000000000000 P 5e9b1f8f888e4119a7d737540089c507 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 06:59:35.944591 11788 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5e9b1f8f888e4119a7d737540089c507 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 06:59:35.944846 11788 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5e9b1f8f888e4119a7d737540089c507 [term 1 LEADER]: Becoming Leader. State: Replica: 5e9b1f8f888e4119a7d737540089c507, State: Running, Role: LEADER
I20260812 06:59:35.944931 19665 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 06:59:35.945210 11788 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5e9b1f8f888e4119a7d737540089c507 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5e9b1f8f888e4119a7d737540089c507" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44407 } }
I20260812 06:59:35.945637 11783 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5e9b1f8f888e4119a7d737540089c507 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 06:59:35.946537 11788 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5e9b1f8f888e4119a7d737540089c507 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5e9b1f8f888e4119a7d737540089c507. Latest consensus state: current_term: 1 leader_uuid: "5e9b1f8f888e4119a7d737540089c507" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5e9b1f8f888e4119a7d737540089c507" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44407 } } }
I20260812 06:59:35.946650 11788 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5e9b1f8f888e4119a7d737540089c507 [sys.catalog]: This master's current role is: LEADER
I20260812 06:59:35.946976 11789 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5e9b1f8f888e4119a7d737540089c507 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5e9b1f8f888e4119a7d737540089c507" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5e9b1f8f888e4119a7d737540089c507" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44407 } } }
I20260812 06:59:35.947080 11789 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5e9b1f8f888e4119a7d737540089c507 [sys.catalog]: This master's current role is: LEADER
W20260812 06:59:35.950680 11811 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 5e9b1f8f888e4119a7d737540089c507: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 06:59:35.950790 11811 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 06:59:35.950872 11803 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 06:59:35.951913 11803 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 06:59:35.954054 11786 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:59:35.947665 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.40:33065 (local address 127.0.0.1:44407)
0812 06:59:35.947783 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0812 06:59:35.947790 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 06:59:35.948910 (+ 1120us) server_negotiation.cc:407] Connection header received
0812 06:59:35.949187 (+ 277us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:59:35.949193 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:59:35.949261 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:59:35.949351 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:59:35.949682 (+ 331us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:59:35.950606 (+ 924us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:59:35.951889 (+ 1283us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:59:35.952254 (+ 365us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:59:35.952374 (+ 120us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:59:35.953466 (+ 1092us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:59:35.953470 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:59:35.953473 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 06:59:35.953489 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:59:35.953520 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 06:59:35.953534 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:59:35.953636 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0812 06:59:35.953841 (+ 205us) server_negotiation.cc:299] Negotiation successful
0812 06:59:35.953887 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260812 06:59:35.958585 11803 catalog_manager.cc:1347] Generated new cluster ID: 937b576b2b144616b4f071761123efdd
I20260812 06:59:35.958671 11803 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 06:59:35.968019 11803 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 06:59:35.969393 11803 catalog_manager.cc:1504] Loading token signing keys...
I20260812 06:59:35.981407 11803 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5e9b1f8f888e4119a7d737540089c507: Generated new TSK 0
I20260812 06:59:35.982311 11803 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 06:59:35.993582 19665 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu with pid 11708
W20260812 06:59:36.002774 11691 connection.cc:535] client connection to 127.0.0.1:44407 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:44407): Connection reset by peer (error 104)
2026-08-12T06:59:35Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/10 (2144 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/11
Loading random data
Initializing database '/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786517687384214-19665-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 06:59:36 dist-test-slave-65mx krb5kdc[11818](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 06:59:36 dist-test-slave-65mx krb5kdc[11818](info): set up 2 sockets
Aug 12 06:59:36 dist-test-slave-65mx krb5kdc[11818](info): commencing operation
krb5kdc: starting...
W20260812 06:59:37.141350 19665 mini_kdc.cc:121] Time spent starting KDC: real 1.123s user 0.005s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786517687384214-19665-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786517687384214-19665-0/krb5kdc/test-user.keytab.
Aug 12 06:59:37 dist-test-slave-65mx krb5kdc[11818](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517977, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T06:59:37Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T06:59:37Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786517687384214-19665-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786517687384214-19665-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260812 06:59:37.284888 19665 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu
/tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786517687384214-19665-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786517687384214-19665-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786517687384214-19665-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786517687384214-19665-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35233
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786517687384214-19665-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:33263
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:35233
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786517687384214-19665-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786517687384214-19665-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786517687384214-19665-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 06:59:37.405699 11833 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:59:37.405992 11833 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:59:37.406036 11833 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 06:59:37.413022 11833 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 06:59:37.413130 11833 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 06:59:37.413168 11833 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 06:59:37.413203 11833 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 06:59:37.413241 11833 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 06:59:37.419782 11833 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:33263
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786517687384214-19665-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786517687384214-19665-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:35233
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786517687384214-19665-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35233
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786517687384214-19665-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11833
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786517687384214-19665-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 06:15:54 UTC on ccycloud.quasar-rnfzce.root.comops.site
build id 4581989
I20260812 06:59:37.421124 11833 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:59:37.422240 11833 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 06:59:37.431447 11840 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:59:37.431445 11843 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:59:37.431480 11841 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:59:37.431823 11833 server_base.cc:1034] running on GCE node
I20260812 06:59:37.432639 11833 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:59:37.433971 11833 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:59:37.435199 11833 hybrid_clock.cc:648] HybridClock initialized: now 1786517977435169 us; error 38 us; skew 500 ppm
Aug 12 06:59:37 dist-test-slave-65mx krb5kdc[11818](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517977, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 06:59:37.438755 11833 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260812 06:59:37.439828 11833 webserver.cc:460] Webserver started at http://127.0.0.1:45211/ using document root <none> and password file <none>
I20260812 06:59:37.440407 11833 fs_manager.cc:359] Metadata directory not provided
I20260812 06:59:37.440629 11833 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:59:37.440894 11833 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 06:59:37.443315 11833 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786517687384214-19665-0/minicluster-data/master-0/data/instance:
uuid: "e7e06135556743afaa3af55a9ced6c15"
format_stamp: "Formatted at 2026-08-12 06:59:37 on dist-test-slave-65mx"
server_key: "c3251d608ce718b069f2f5cc69f5416a"
server_key_iv: "366a6bff8e5636a6bfef8859bc580bc3"
server_key_version: "encryptionkey@0"
I20260812 06:59:37.443936 11833 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786517687384214-19665-0/minicluster-data/master-0/wal/instance:
uuid: "e7e06135556743afaa3af55a9ced6c15"
format_stamp: "Formatted at 2026-08-12 06:59:37 on dist-test-slave-65mx"
server_key: "c3251d608ce718b069f2f5cc69f5416a"
server_key_iv: "366a6bff8e5636a6bfef8859bc580bc3"
server_key_version: "encryptionkey@0"
I20260812 06:59:37.448387 11833 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.007s sys 0.000s
I20260812 06:59:37.451509 11850 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 06:59:37.452993 11833 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 06:59:37.453161 11833 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786517687384214-19665-0/minicluster-data/master-0/data,/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786517687384214-19665-0/minicluster-data/master-0/wal
uuid: "e7e06135556743afaa3af55a9ced6c15"
format_stamp: "Formatted at 2026-08-12 06:59:37 on dist-test-slave-65mx"
server_key: "c3251d608ce718b069f2f5cc69f5416a"
server_key_iv: "366a6bff8e5636a6bfef8859bc580bc3"
server_key_version: "encryptionkey@0"
I20260812 06:59:37.453286 11833 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786517687384214-19665-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786517687384214-19665-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786517687384214-19665-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:59:37.458331 11833 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 06:59:37.461985 11833 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 06:59:37.462245 11833 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:59:37.470592 11833 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:35233
I20260812 06:59:37.470661 11907 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:35233 every 8 connection(s)
I20260812 06:59:37.471704 11833 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786517687384214-19665-0/minicluster-data/master-0/data/info.pb
I20260812 06:59:37.473558 19665 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu as pid 11833
I20260812 06:59:37.473716 19665 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786517687384214-19665-0/minicluster-data/master-0/wal/instance
I20260812 06:59:37.474035 19665 external_mini_cluster.cc:1428] Setting key e90f374aa6cd329a43d8dfe643df6b40
I20260812 06:59:37.475292 11908 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 06:59:37.479730 11911 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:59:37.475586 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.40:37573 (local address 127.0.0.1:35233)
0812 06:59:37.476091 (+ 505us) server_negotiation.cc:206] Beginning negotiation
0812 06:59:37.476107 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0812 06:59:37.476146 (+ 39us) server_negotiation.cc:407] Connection header received
0812 06:59:37.476924 (+ 778us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:59:37.476954 (+ 30us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:59:37.477250 (+ 296us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:59:37.477612 (+ 362us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:59:37.478449 (+ 837us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:59:37.478453 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:59:37.478465 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 06:59:37.478476 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:59:37.478531 (+ 55us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 06:59:37.478550 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:59:37.478722 (+ 172us) server_negotiation.cc:1035] Waiting for connection context
0812 06:59:37.478843 (+ 121us) server_negotiation.cc:299] Negotiation successful
0812 06:59:37.479004 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":356,"thread_start_us":158,"threads_started":1}
I20260812 06:59:37.482090 11908 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e7e06135556743afaa3af55a9ced6c15: Bootstrap starting.
I20260812 06:59:37.483255 19665 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 06:59:37.485061 11908 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e7e06135556743afaa3af55a9ced6c15: Neither blocks nor log segments found. Creating new log.
I20260812 06:59:37.486032 11908 log.cc:824] T 00000000000000000000000000000000 P e7e06135556743afaa3af55a9ced6c15: Log is configured to *not* fsync() on all Append() calls
I20260812 06:59:37.487799 11911 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:59:37.485691 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.40:44583 (local address 127.0.0.1:35233)
0812 06:59:37.485840 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0812 06:59:37.485845 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 06:59:37.486075 (+ 230us) server_negotiation.cc:407] Connection header received
0812 06:59:37.486263 (+ 188us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:59:37.486266 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:59:37.486301 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:59:37.486373 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:59:37.487259 (+ 886us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:59:37.487262 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:59:37.487264 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 06:59:37.487274 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:59:37.487301 (+ 27us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 06:59:37.487311 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:59:37.487385 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0812 06:59:37.487558 (+ 173us) server_negotiation.cc:299] Negotiation successful
0812 06:59:37.487637 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260812 06:59:37.488725 11908 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e7e06135556743afaa3af55a9ced6c15: No bootstrap required, opened a new log
I20260812 06:59:37.492659 11908 raft_consensus.cc:348] T 00000000000000000000000000000000 P e7e06135556743afaa3af55a9ced6c15 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e7e06135556743afaa3af55a9ced6c15" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35233 } }
I20260812 06:59:37.492952 11908 raft_consensus.cc:374] T 00000000000000000000000000000000 P e7e06135556743afaa3af55a9ced6c15 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 06:59:37.493057 11908 raft_consensus.cc:729] T 00000000000000000000000000000000 P e7e06135556743afaa3af55a9ced6c15 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e7e06135556743afaa3af55a9ced6c15, State: Initialized, Role: FOLLOWER
I20260812 06:59:37.493551 11908 consensus_queue.cc:260] T 00000000000000000000000000000000 P e7e06135556743afaa3af55a9ced6c15 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e7e06135556743afaa3af55a9ced6c15" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35233 } }
I20260812 06:59:37.493713 11908 raft_consensus.cc:388] T 00000000000000000000000000000000 P e7e06135556743afaa3af55a9ced6c15 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 06:59:37.493806 11908 raft_consensus.cc:482] T 00000000000000000000000000000000 P e7e06135556743afaa3af55a9ced6c15 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 06:59:37.493929 11908 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e7e06135556743afaa3af55a9ced6c15 [term 0 FOLLOWER]: Advancing to term 1
I20260812 06:59:37.494956 11908 raft_consensus.cc:504] T 00000000000000000000000000000000 P e7e06135556743afaa3af55a9ced6c15 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e7e06135556743afaa3af55a9ced6c15" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35233 } }
I20260812 06:59:37.495329 11908 leader_election.cc:302] T 00000000000000000000000000000000 P e7e06135556743afaa3af55a9ced6c15 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e7e06135556743afaa3af55a9ced6c15; no voters:
I20260812 06:59:37.495718 11908 leader_election.cc:288] T 00000000000000000000000000000000 P e7e06135556743afaa3af55a9ced6c15 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 06:59:37.495904 11919 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e7e06135556743afaa3af55a9ced6c15 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 06:59:37.496263 11919 raft_consensus.cc:686] T 00000000000000000000000000000000 P e7e06135556743afaa3af55a9ced6c15 [term 1 LEADER]: Becoming Leader. State: Replica: e7e06135556743afaa3af55a9ced6c15, State: Running, Role: LEADER
I20260812 06:59:37.496760 11919 consensus_queue.cc:237] T 00000000000000000000000000000000 P e7e06135556743afaa3af55a9ced6c15 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e7e06135556743afaa3af55a9ced6c15" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35233 } }
I20260812 06:59:37.497411 11908 sys_catalog.cc:564] T 00000000000000000000000000000000 P e7e06135556743afaa3af55a9ced6c15 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 06:59:37.498768 11921 sys_catalog.cc:455] T 00000000000000000000000000000000 P e7e06135556743afaa3af55a9ced6c15 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e7e06135556743afaa3af55a9ced6c15. Latest consensus state: current_term: 1 leader_uuid: "e7e06135556743afaa3af55a9ced6c15" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e7e06135556743afaa3af55a9ced6c15" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35233 } } }
I20260812 06:59:37.498919 11921 sys_catalog.cc:458] T 00000000000000000000000000000000 P e7e06135556743afaa3af55a9ced6c15 [sys.catalog]: This master's current role is: LEADER
I20260812 06:59:37.499281 11920 sys_catalog.cc:455] T 00000000000000000000000000000000 P e7e06135556743afaa3af55a9ced6c15 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e7e06135556743afaa3af55a9ced6c15" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e7e06135556743afaa3af55a9ced6c15" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35233 } } }
I20260812 06:59:37.499380 11920 sys_catalog.cc:458] T 00000000000000000000000000000000 P e7e06135556743afaa3af55a9ced6c15 [sys.catalog]: This master's current role is: LEADER
W20260812 06:59:37.502398 11938 catalog_manager.cc:1558] T 00000000000000000000000000000000 P e7e06135556743afaa3af55a9ced6c15: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 06:59:37.502528 11938 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 06:59:37.502614 11930 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 06:59:37.503336 11930 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 06:59:37.509325 11930 catalog_manager.cc:1347] Generated new cluster ID: dcbe5f2048e94bf79e1f596855b14db7
I20260812 06:59:37.509430 11930 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 06:59:37.521243 11930 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 06:59:37.521752 11930 catalog_manager.cc:1504] Loading token signing keys...
I20260812 06:59:37.531450 11930 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e7e06135556743afaa3af55a9ced6c15: Generated new TSK 0
I20260812 06:59:37.532270 11930 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 06:59:37.539629 19665 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu with pid 11833
2026-08-12T06:59:37Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/11 (1545 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/12
Loading random data
Initializing database '/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786517687384214-19665-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 06:59:37 dist-test-slave-65mx krb5kdc[11944](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 06:59:37 dist-test-slave-65mx krb5kdc[11944](info): set up 2 sockets
Aug 12 06:59:37 dist-test-slave-65mx krb5kdc[11944](info): commencing operation
krb5kdc: starting...
W20260812 06:59:38.679287 19665 mini_kdc.cc:121] Time spent starting KDC: real 1.116s user 0.002s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786517687384214-19665-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786517687384214-19665-0/krb5kdc/test-user.keytab.
Aug 12 06:59:38 dist-test-slave-65mx krb5kdc[11944](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517978, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T06:59:38Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T06:59:38Z Disabled control of system clock
WARNING: no policy specified for kudu/127.19.52.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.52.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.52.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786517687384214-19665-0/krb5kdc/kudu_127.19.52.126.keytab.
Entry for principal kudu/127.19.52.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786517687384214-19665-0/krb5kdc/kudu_127.19.52.126.keytab.
I20260812 06:59:38.822206 19665 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu
/tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786517687384214-19665-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786517687384214-19665-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786517687384214-19665-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786517687384214-19665-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.52.126:42433
--webserver_interface=127.19.52.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786517687384214-19665-0/krb5kdc/kudu_127.19.52.126.keytab
--principal=kudu/127.19.52.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.52.84:43753
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.19.52.126:42433
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786517687384214-19665-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786517687384214-19665-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786517687384214-19665-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 06:59:38.943212 11958 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:59:38.943610 11958 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:59:38.943703 11958 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 06:59:38.948019 11958 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 06:59:38.948146 11958 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 06:59:38.948184 11958 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 06:59:38.948230 11958 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 06:59:38.948263 11958 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 06:59:38.954128 11958 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.52.84:43753
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786517687384214-19665-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786517687384214-19665-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.19.52.126:42433
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786517687384214-19665-0/krb5kdc/kudu_127.19.52.126.keytab
--principal=kudu/127.19.52.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.52.126:42433
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786517687384214-19665-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.19.52.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11958
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786517687384214-19665-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 06:15:54 UTC on ccycloud.quasar-rnfzce.root.comops.site
build id 4581989
I20260812 06:59:38.955440 11958 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:59:38.956568 11958 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 06:59:38.966650 11968 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:59:38.966600 11964 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:59:38.966645 11965 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:59:38.966879 11958 server_base.cc:1034] running on GCE node
I20260812 06:59:38.967509 11958 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:59:38.968743 11958 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:59:38.969913 11958 hybrid_clock.cc:648] HybridClock initialized: now 1786517978969894 us; error 35 us; skew 500 ppm
Aug 12 06:59:38 dist-test-slave-65mx krb5kdc[11944](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517978, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.52.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 06:59:38.973132 11958 init.cc:377] Logged in from keytab as kudu/127.19.52.126@KRBTEST.COM (short username kudu)
I20260812 06:59:38.974181 11958 webserver.cc:460] Webserver started at http://127.19.52.126:40099/ using document root <none> and password file <none>
I20260812 06:59:38.974931 11958 fs_manager.cc:359] Metadata directory not provided
I20260812 06:59:38.975008 11958 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:59:38.975293 11958 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 06:59:38.977653 11958 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786517687384214-19665-0/minicluster-data/master-0/data/instance:
uuid: "fa4f17a536124ed9814b7d38e4274dbf"
format_stamp: "Formatted at 2026-08-12 06:59:38 on dist-test-slave-65mx"
server_key: "2416630bf1655b1e0389d8fa534e665d"
server_key_iv: "a999e232a962ef5acc13d666ef300839"
server_key_version: "encryptionkey@0"
I20260812 06:59:38.978260 11958 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786517687384214-19665-0/minicluster-data/master-0/wal/instance:
uuid: "fa4f17a536124ed9814b7d38e4274dbf"
format_stamp: "Formatted at 2026-08-12 06:59:38 on dist-test-slave-65mx"
server_key: "2416630bf1655b1e0389d8fa534e665d"
server_key_iv: "a999e232a962ef5acc13d666ef300839"
server_key_version: "encryptionkey@0"
I20260812 06:59:38.982366 11958 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.004s
I20260812 06:59:38.985414 11977 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 06:59:38.986924 11958 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260812 06:59:38.987110 11958 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786517687384214-19665-0/minicluster-data/master-0/data,/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786517687384214-19665-0/minicluster-data/master-0/wal
uuid: "fa4f17a536124ed9814b7d38e4274dbf"
format_stamp: "Formatted at 2026-08-12 06:59:38 on dist-test-slave-65mx"
server_key: "2416630bf1655b1e0389d8fa534e665d"
server_key_iv: "a999e232a962ef5acc13d666ef300839"
server_key_version: "encryptionkey@0"
I20260812 06:59:38.987258 11958 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786517687384214-19665-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786517687384214-19665-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786517687384214-19665-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:59:38.997642 11958 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 06:59:39.000965 11958 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 06:59:39.001190 11958 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:59:39.008737 11958 rpc_server.cc:307] RPC server started. Bound to: 127.19.52.126:42433
I20260812 06:59:39.008767 12032 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.52.126:42433 every 8 connection(s)
I20260812 06:59:39.009720 11958 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786517687384214-19665-0/minicluster-data/master-0/data/info.pb
I20260812 06:59:39.010569 19665 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu as pid 11958
I20260812 06:59:39.010731 19665 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786517687384214-19665-0/minicluster-data/master-0/wal/instance
I20260812 06:59:39.011057 19665 external_mini_cluster.cc:1428] Setting key 0e3c4921db4f713429a3f2d079644c77
I20260812 06:59:39.013188 12033 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 06:59:39.019021 12033 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fa4f17a536124ed9814b7d38e4274dbf: Bootstrap starting.
Aug 12 06:59:39 dist-test-slave-65mx krb5kdc[11944](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517978, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.19.52.126@KRBTEST.COM
I20260812 06:59:39.021639 12033 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P fa4f17a536124ed9814b7d38e4274dbf: Neither blocks nor log segments found. Creating new log.
I20260812 06:59:39.022557 12033 log.cc:824] T 00000000000000000000000000000000 P fa4f17a536124ed9814b7d38e4274dbf: Log is configured to *not* fsync() on all Append() calls
I20260812 06:59:39.025091 12033 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fa4f17a536124ed9814b7d38e4274dbf: No bootstrap required, opened a new log
I20260812 06:59:39.026644 12036 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:59:39.012637 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50977 (local address 127.19.52.126:42433)
0812 06:59:39.013141 (+ 504us) server_negotiation.cc:206] Beginning negotiation
0812 06:59:39.013161 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0812 06:59:39.013198 (+ 37us) server_negotiation.cc:407] Connection header received
0812 06:59:39.013885 (+ 687us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:59:39.013902 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:59:39.014247 (+ 345us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:59:39.014586 (+ 339us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:59:39.015465 (+ 879us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:59:39.016470 (+ 1005us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:59:39.017386 (+ 916us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:59:39.017876 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:59:39.018068 (+ 192us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:59:39.021052 (+ 2984us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:59:39.021057 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:59:39.021072 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:59:39.021104 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:59:39.023260 (+ 2156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:59:39.023953 (+ 693us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:59:39.023958 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:59:39.023963 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:59:39.024116 (+ 153us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:59:39.024527 (+ 411us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:59:39.024530 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:59:39.024531 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:59:39.025065 (+ 534us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:59:39.025288 (+ 223us) server_negotiation.cc:1035] Waiting for connection context
0812 06:59:39.025586 (+ 298us) server_negotiation.cc:299] Negotiation successful
0812 06:59:39.025821 (+ 235us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":323,"thread_start_us":163,"threads_started":1}
I20260812 06:59:39.029141 12033 raft_consensus.cc:348] T 00000000000000000000000000000000 P fa4f17a536124ed9814b7d38e4274dbf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fa4f17a536124ed9814b7d38e4274dbf" member_type: VOTER last_known_addr { host: "127.19.52.126" port: 42433 } }
I20260812 06:59:39.029403 12033 raft_consensus.cc:374] T 00000000000000000000000000000000 P fa4f17a536124ed9814b7d38e4274dbf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 06:59:39.029481 12033 raft_consensus.cc:729] T 00000000000000000000000000000000 P fa4f17a536124ed9814b7d38e4274dbf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fa4f17a536124ed9814b7d38e4274dbf, State: Initialized, Role: FOLLOWER
I20260812 06:59:39.029953 19665 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 06:59:39.029978 12033 consensus_queue.cc:260] T 00000000000000000000000000000000 P fa4f17a536124ed9814b7d38e4274dbf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fa4f17a536124ed9814b7d38e4274dbf" member_type: VOTER last_known_addr { host: "127.19.52.126" port: 42433 } }
I20260812 06:59:39.030166 12033 raft_consensus.cc:388] T 00000000000000000000000000000000 P fa4f17a536124ed9814b7d38e4274dbf [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 06:59:39.030251 12033 raft_consensus.cc:482] T 00000000000000000000000000000000 P fa4f17a536124ed9814b7d38e4274dbf [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 06:59:39.030354 12033 raft_consensus.cc:3037] T 00000000000000000000000000000000 P fa4f17a536124ed9814b7d38e4274dbf [term 0 FOLLOWER]: Advancing to term 1
I20260812 06:59:39.031435 12033 raft_consensus.cc:504] T 00000000000000000000000000000000 P fa4f17a536124ed9814b7d38e4274dbf [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fa4f17a536124ed9814b7d38e4274dbf" member_type: VOTER last_known_addr { host: "127.19.52.126" port: 42433 } }
I20260812 06:59:39.031819 12033 leader_election.cc:302] T 00000000000000000000000000000000 P fa4f17a536124ed9814b7d38e4274dbf [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: fa4f17a536124ed9814b7d38e4274dbf; no voters:
I20260812 06:59:39.032169 12033 leader_election.cc:288] T 00000000000000000000000000000000 P fa4f17a536124ed9814b7d38e4274dbf [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 06:59:39.032518 12043 raft_consensus.cc:2781] T 00000000000000000000000000000000 P fa4f17a536124ed9814b7d38e4274dbf [term 1 FOLLOWER]: Leader election won for term 1
I20260812 06:59:39.032810 12043 raft_consensus.cc:686] T 00000000000000000000000000000000 P fa4f17a536124ed9814b7d38e4274dbf [term 1 LEADER]: Becoming Leader. State: Replica: fa4f17a536124ed9814b7d38e4274dbf, State: Running, Role: LEADER
I20260812 06:59:39.033186 12043 consensus_queue.cc:237] T 00000000000000000000000000000000 P fa4f17a536124ed9814b7d38e4274dbf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fa4f17a536124ed9814b7d38e4274dbf" member_type: VOTER last_known_addr { host: "127.19.52.126" port: 42433 } }
I20260812 06:59:39.033650 12033 sys_catalog.cc:564] T 00000000000000000000000000000000 P fa4f17a536124ed9814b7d38e4274dbf [sys.catalog]: configured and running, proceeding with master startup.
I20260812 06:59:39.034444 12043 sys_catalog.cc:455] T 00000000000000000000000000000000 P fa4f17a536124ed9814b7d38e4274dbf [sys.catalog]: SysCatalogTable state changed. Reason: New leader fa4f17a536124ed9814b7d38e4274dbf. Latest consensus state: current_term: 1 leader_uuid: "fa4f17a536124ed9814b7d38e4274dbf" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fa4f17a536124ed9814b7d38e4274dbf" member_type: VOTER last_known_addr { host: "127.19.52.126" port: 42433 } } }
I20260812 06:59:39.034658 12043 sys_catalog.cc:458] T 00000000000000000000000000000000 P fa4f17a536124ed9814b7d38e4274dbf [sys.catalog]: This master's current role is: LEADER
I20260812 06:59:39.034835 12045 sys_catalog.cc:455] T 00000000000000000000000000000000 P fa4f17a536124ed9814b7d38e4274dbf [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "fa4f17a536124ed9814b7d38e4274dbf" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fa4f17a536124ed9814b7d38e4274dbf" member_type: VOTER last_known_addr { host: "127.19.52.126" port: 42433 } } }
I20260812 06:59:39.034947 12045 sys_catalog.cc:458] T 00000000000000000000000000000000 P fa4f17a536124ed9814b7d38e4274dbf [sys.catalog]: This master's current role is: LEADER
W20260812 06:59:39.038939 12061 catalog_manager.cc:1558] T 00000000000000000000000000000000 P fa4f17a536124ed9814b7d38e4274dbf: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 06:59:39.039044 12061 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 06:59:39.039124 12054 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 06:59:39.040195 12054 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 06:59:39.042071 12036 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:59:39.032248 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57669 (local address 127.19.52.126:42433)
0812 06:59:39.032390 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0812 06:59:39.032396 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 06:59:39.032615 (+ 219us) server_negotiation.cc:407] Connection header received
0812 06:59:39.032741 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:59:39.032745 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:59:39.032798 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:59:39.032871 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:59:39.033725 (+ 854us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:59:39.034382 (+ 657us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:59:39.035574 (+ 1192us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:59:39.035919 (+ 345us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:59:39.036023 (+ 104us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:59:39.038071 (+ 2048us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:59:39.038074 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:59:39.038077 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:59:39.038104 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:59:39.039877 (+ 1773us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:59:39.040510 (+ 633us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:59:39.040514 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:59:39.040516 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:59:39.040681 (+ 165us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:59:39.041075 (+ 394us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:59:39.041079 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:59:39.041080 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:59:39.041305 (+ 225us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:59:39.041454 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0812 06:59:39.041773 (+ 319us) server_negotiation.cc:299] Negotiation successful
0812 06:59:39.041895 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260812 06:59:39.047339 12054 catalog_manager.cc:1347] Generated new cluster ID: 5f356e67a4e545c9a242387e9af55c12
I20260812 06:59:39.047432 12054 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 06:59:39.064325 12054 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 06:59:39.065470 12054 catalog_manager.cc:1504] Loading token signing keys...
I20260812 06:59:39.077035 12054 catalog_manager.cc:5868] T 00000000000000000000000000000000 P fa4f17a536124ed9814b7d38e4274dbf: Generated new TSK 0
I20260812 06:59:39.077935 12054 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 06:59:39.084853 19665 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu with pid 11958
W20260812 06:59:39.094079 11942 connection.cc:535] client connection to 127.19.52.126:42433 recv error: Network error: failed to read from TLS socket (remote: 127.19.52.126:42433): Connection reset by peer (error 104)
2026-08-12T06:59:39Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/12 (1545 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/13
Loading random data
Initializing database '/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786517687384214-19665-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 06:59:39 dist-test-slave-65mx krb5kdc[12068](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 06:59:39 dist-test-slave-65mx krb5kdc[12068](info): set up 2 sockets
Aug 12 06:59:39 dist-test-slave-65mx krb5kdc[12068](info): commencing operation
krb5kdc: starting...
W20260812 06:59:40.230738 19665 mini_kdc.cc:121] Time spent starting KDC: real 1.122s user 0.000s sys 0.009s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786517687384214-19665-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786517687384214-19665-0/krb5kdc/test-user.keytab.
Aug 12 06:59:40 dist-test-slave-65mx krb5kdc[12068](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517980, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T06:59:40Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T06:59:40Z Disabled control of system clock
WARNING: no policy specified for kudu/127.19.52.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.52.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.52.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786517687384214-19665-0/krb5kdc/kudu_127.19.52.126.keytab.
Entry for principal kudu/127.19.52.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786517687384214-19665-0/krb5kdc/kudu_127.19.52.126.keytab.
I20260812 06:59:40.372954 19665 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu
/tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786517687384214-19665-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786517687384214-19665-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786517687384214-19665-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786517687384214-19665-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.52.126:37611
--webserver_interface=127.19.52.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786517687384214-19665-0/krb5kdc/kudu_127.19.52.126.keytab
--principal=kudu/127.19.52.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.52.84:39205
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.19.52.126:37611
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786517687384214-19665-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786517687384214-19665-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786517687384214-19665-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 06:59:40.489490 12082 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:59:40.489837 12082 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:59:40.489935 12082 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 06:59:40.494160 12082 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 06:59:40.494266 12082 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 06:59:40.494306 12082 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 06:59:40.494345 12082 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 06:59:40.494377 12082 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 06:59:40.500528 12082 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.52.84:39205
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786517687384214-19665-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786517687384214-19665-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.19.52.126:37611
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786517687384214-19665-0/krb5kdc/kudu_127.19.52.126.keytab
--principal=kudu/127.19.52.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.52.126:37611
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786517687384214-19665-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.19.52.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12082
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786517687384214-19665-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 06:15:54 UTC on ccycloud.quasar-rnfzce.root.comops.site
build id 4581989
I20260812 06:59:40.501775 12082 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:59:40.502825 12082 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 06:59:40.512001 12092 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:59:40.512028 12089 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:59:40.512039 12082 server_base.cc:1034] running on GCE node
W20260812 06:59:40.512007 12090 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:59:40.513154 12082 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:59:40.514698 12082 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:59:40.515902 12082 hybrid_clock.cc:648] HybridClock initialized: now 1786517980515883 us; error 43 us; skew 500 ppm
Aug 12 06:59:40 dist-test-slave-65mx krb5kdc[12068](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517980, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.52.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 06:59:40.518846 12082 init.cc:377] Logged in from keytab as kudu/127.19.52.126@KRBTEST.COM (short username kudu)
I20260812 06:59:40.519707 12082 webserver.cc:460] Webserver started at http://127.19.52.126:35343/ using document root <none> and password file <none>
I20260812 06:59:40.520274 12082 fs_manager.cc:359] Metadata directory not provided
I20260812 06:59:40.520344 12082 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:59:40.520514 12082 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 06:59:40.522641 12082 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786517687384214-19665-0/minicluster-data/master-0/data/instance:
uuid: "1694c5d60294415598d476ea6cbc181d"
format_stamp: "Formatted at 2026-08-12 06:59:40 on dist-test-slave-65mx"
server_key: "0314d5a7939cb492637dcc16407e8db6"
server_key_iv: "5b67c932a42da94ca709f00e46df847a"
server_key_version: "encryptionkey@0"
I20260812 06:59:40.523173 12082 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786517687384214-19665-0/minicluster-data/master-0/wal/instance:
uuid: "1694c5d60294415598d476ea6cbc181d"
format_stamp: "Formatted at 2026-08-12 06:59:40 on dist-test-slave-65mx"
server_key: "0314d5a7939cb492637dcc16407e8db6"
server_key_iv: "5b67c932a42da94ca709f00e46df847a"
server_key_version: "encryptionkey@0"
I20260812 06:59:40.527177 12082 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.003s
I20260812 06:59:40.529955 12100 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 06:59:40.531487 12082 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260812 06:59:40.531633 12082 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786517687384214-19665-0/minicluster-data/master-0/data,/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786517687384214-19665-0/minicluster-data/master-0/wal
uuid: "1694c5d60294415598d476ea6cbc181d"
format_stamp: "Formatted at 2026-08-12 06:59:40 on dist-test-slave-65mx"
server_key: "0314d5a7939cb492637dcc16407e8db6"
server_key_iv: "5b67c932a42da94ca709f00e46df847a"
server_key_version: "encryptionkey@0"
I20260812 06:59:40.531754 12082 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786517687384214-19665-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786517687384214-19665-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786517687384214-19665-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:59:40.549723 12082 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 06:59:40.553861 12082 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 06:59:40.554126 12082 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:59:40.562252 12082 rpc_server.cc:307] RPC server started. Bound to: 127.19.52.126:37611
I20260812 06:59:40.562284 12161 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.52.126:37611 every 8 connection(s)
I20260812 06:59:40.563359 12082 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786517687384214-19665-0/minicluster-data/master-0/data/info.pb
I20260812 06:59:40.566371 12162 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 06:59:40.571216 19665 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu as pid 12082
I20260812 06:59:40.571349 19665 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786517687384214-19665-0/minicluster-data/master-0/wal/instance
I20260812 06:59:40.571678 19665 external_mini_cluster.cc:1428] Setting key 293eff8db9b69eb84957e63c6a54a79c
I20260812 06:59:40.572371 12162 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1694c5d60294415598d476ea6cbc181d: Bootstrap starting.
I20260812 06:59:40.574977 12162 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1694c5d60294415598d476ea6cbc181d: Neither blocks nor log segments found. Creating new log.
I20260812 06:59:40.575934 12162 log.cc:824] T 00000000000000000000000000000000 P 1694c5d60294415598d476ea6cbc181d: Log is configured to *not* fsync() on all Append() calls
I20260812 06:59:40.578454 12162 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1694c5d60294415598d476ea6cbc181d: No bootstrap required, opened a new log
I20260812 06:59:40.581893 12165 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:59:40.573199 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57685 (local address 127.19.52.126:37611)
0812 06:59:40.573711 (+ 512us) server_negotiation.cc:206] Beginning negotiation
0812 06:59:40.573728 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0812 06:59:40.573767 (+ 39us) server_negotiation.cc:407] Connection header received
0812 06:59:40.574434 (+ 667us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:59:40.574453 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:59:40.574914 (+ 461us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:59:40.575286 (+ 372us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:59:40.575659 (+ 373us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:59:40.576968 (+ 1309us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:59:40.578251 (+ 1283us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:59:40.578870 (+ 619us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:59:40.579065 (+ 195us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:59:40.580028 (+ 963us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:59:40.580033 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:59:40.580046 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 06:59:40.580475 (+ 429us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:59:40.580534 (+ 59us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 06:59:40.580620 (+ 86us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:59:40.580728 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0812 06:59:40.580940 (+ 212us) server_negotiation.cc:299] Negotiation successful
0812 06:59:40.581100 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":359,"thread_start_us":161,"threads_started":1}
I20260812 06:59:40.582705 12162 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1694c5d60294415598d476ea6cbc181d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1694c5d60294415598d476ea6cbc181d" member_type: VOTER last_known_addr { host: "127.19.52.126" port: 37611 } }
I20260812 06:59:40.582989 12162 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1694c5d60294415598d476ea6cbc181d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 06:59:40.583050 12162 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1694c5d60294415598d476ea6cbc181d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1694c5d60294415598d476ea6cbc181d, State: Initialized, Role: FOLLOWER
I20260812 06:59:40.583515 12162 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1694c5d60294415598d476ea6cbc181d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1694c5d60294415598d476ea6cbc181d" member_type: VOTER last_known_addr { host: "127.19.52.126" port: 37611 } }
I20260812 06:59:40.583670 12162 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1694c5d60294415598d476ea6cbc181d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 06:59:40.583761 12162 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1694c5d60294415598d476ea6cbc181d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 06:59:40.583873 12162 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1694c5d60294415598d476ea6cbc181d [term 0 FOLLOWER]: Advancing to term 1
I20260812 06:59:40.585057 19665 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 06:59:40.585217 12162 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1694c5d60294415598d476ea6cbc181d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1694c5d60294415598d476ea6cbc181d" member_type: VOTER last_known_addr { host: "127.19.52.126" port: 37611 } }
I20260812 06:59:40.585583 12162 leader_election.cc:302] T 00000000000000000000000000000000 P 1694c5d60294415598d476ea6cbc181d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1694c5d60294415598d476ea6cbc181d; no voters:
I20260812 06:59:40.585965 12162 leader_election.cc:288] T 00000000000000000000000000000000 P 1694c5d60294415598d476ea6cbc181d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 06:59:40.586081 12167 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1694c5d60294415598d476ea6cbc181d [term 1 FOLLOWER]: Leader election won for term 1
I20260812 06:59:40.586387 12167 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1694c5d60294415598d476ea6cbc181d [term 1 LEADER]: Becoming Leader. State: Replica: 1694c5d60294415598d476ea6cbc181d, State: Running, Role: LEADER
I20260812 06:59:40.586767 12167 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1694c5d60294415598d476ea6cbc181d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1694c5d60294415598d476ea6cbc181d" member_type: VOTER last_known_addr { host: "127.19.52.126" port: 37611 } }
I20260812 06:59:40.587409 12162 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1694c5d60294415598d476ea6cbc181d [sys.catalog]: configured and running, proceeding with master startup.
I20260812 06:59:40.588027 12168 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1694c5d60294415598d476ea6cbc181d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1694c5d60294415598d476ea6cbc181d. Latest consensus state: current_term: 1 leader_uuid: "1694c5d60294415598d476ea6cbc181d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1694c5d60294415598d476ea6cbc181d" member_type: VOTER last_known_addr { host: "127.19.52.126" port: 37611 } } }
I20260812 06:59:40.588156 12168 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1694c5d60294415598d476ea6cbc181d [sys.catalog]: This master's current role is: LEADER
I20260812 06:59:40.588521 12169 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1694c5d60294415598d476ea6cbc181d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1694c5d60294415598d476ea6cbc181d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1694c5d60294415598d476ea6cbc181d" member_type: VOTER last_known_addr { host: "127.19.52.126" port: 37611 } } }
I20260812 06:59:40.588619 12178 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 06:59:40.588671 12169 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1694c5d60294415598d476ea6cbc181d [sys.catalog]: This master's current role is: LEADER
I20260812 06:59:40.591730 12178 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 06:59:40.595736 12165 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:59:40.587458 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52527 (local address 127.19.52.126:37611)
0812 06:59:40.587570 (+ 112us) server_negotiation.cc:206] Beginning negotiation
0812 06:59:40.587574 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 06:59:40.591313 (+ 3739us) server_negotiation.cc:407] Connection header received
0812 06:59:40.591668 (+ 355us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:59:40.591671 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:59:40.591721 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:59:40.591813 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:59:40.592198 (+ 385us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:59:40.593011 (+ 813us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:59:40.593856 (+ 845us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:59:40.594135 (+ 279us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:59:40.594262 (+ 127us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:59:40.595213 (+ 951us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:59:40.595217 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:59:40.595219 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 06:59:40.595235 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:59:40.595267 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 06:59:40.595278 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:59:40.595376 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0812 06:59:40.595523 (+ 147us) server_negotiation.cc:299] Negotiation successful
0812 06:59:40.595571 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260812 06:59:40.598524 12178 catalog_manager.cc:1347] Generated new cluster ID: f22013db4f0140c1b45afad6f89da701
I20260812 06:59:40.598614 12178 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 06:59:40.610194 12178 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 06:59:40.611582 12178 catalog_manager.cc:1504] Loading token signing keys...
I20260812 06:59:40.622478 12178 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1694c5d60294415598d476ea6cbc181d: Generated new TSK 0
I20260812 06:59:40.623399 12178 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 06:59:40.633036 19665 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu with pid 12082
W20260812 06:59:40.642632 12066 connection.cc:535] client connection to 127.19.52.126:37611 recv error: Network error: failed to read from TLS socket (remote: 127.19.52.126:37611): Connection reset by peer (error 104)
2026-08-12T06:59:40Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/13 (1548 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/14
Loading random data
Initializing database '/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786517687384214-19665-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 06:59:40 dist-test-slave-65mx krb5kdc[12196](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 06:59:40 dist-test-slave-65mx krb5kdc[12196](info): set up 2 sockets
Aug 12 06:59:40 dist-test-slave-65mx krb5kdc[12196](info): commencing operation
krb5kdc: starting...
W20260812 06:59:41.784389 19665 mini_kdc.cc:121] Time spent starting KDC: real 1.127s user 0.004s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786517687384214-19665-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786517687384214-19665-0/krb5kdc/test-user.keytab.
Aug 12 06:59:41 dist-test-slave-65mx krb5kdc[12196](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517981, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T06:59:41Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T06:59:41Z Disabled control of system clock
WARNING: no policy specified for kudu/127.19.52.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.52.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.52.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786517687384214-19665-0/krb5kdc/kudu_127.19.52.126.keytab.
Entry for principal kudu/127.19.52.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786517687384214-19665-0/krb5kdc/kudu_127.19.52.126.keytab.
I20260812 06:59:41.923507 19665 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu
/tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786517687384214-19665-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786517687384214-19665-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786517687384214-19665-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786517687384214-19665-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.52.126:36055
--webserver_interface=127.19.52.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786517687384214-19665-0/krb5kdc/kudu_127.19.52.126.keytab
--principal=kudu/127.19.52.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.52.84:39895
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.19.52.126:36055
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786517687384214-19665-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786517687384214-19665-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786517687384214-19665-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 06:59:42.040901 12209 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:59:42.041189 12209 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:59:42.041232 12209 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 06:59:42.045312 12209 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 06:59:42.045377 12209 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 06:59:42.045395 12209 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 06:59:42.045437 12209 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 06:59:42.045475 12209 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 06:59:42.051131 12209 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.52.84:39895
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786517687384214-19665-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786517687384214-19665-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.19.52.126:36055
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786517687384214-19665-0/krb5kdc/kudu_127.19.52.126.keytab
--principal=kudu/127.19.52.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.52.126:36055
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786517687384214-19665-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.19.52.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12209
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786517687384214-19665-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 06:15:54 UTC on ccycloud.quasar-rnfzce.root.comops.site
build id 4581989
I20260812 06:59:42.052277 12209 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:59:42.053349 12209 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 06:59:42.062053 12219 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:59:42.062130 12209 server_base.cc:1034] running on GCE node
W20260812 06:59:42.062076 12216 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:59:42.062065 12215 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:59:42.063097 12209 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:59:42.064306 12209 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:59:42.065528 12209 hybrid_clock.cc:648] HybridClock initialized: now 1786517982065479 us; error 52 us; skew 500 ppm
Aug 12 06:59:42 dist-test-slave-65mx krb5kdc[12196](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517982, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.52.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 06:59:42.072512 12209 init.cc:377] Logged in from keytab as kudu/127.19.52.126@KRBTEST.COM (short username kudu)
I20260812 06:59:42.073508 12209 webserver.cc:460] Webserver started at http://127.19.52.126:44379/ using document root <none> and password file <none>
I20260812 06:59:42.074126 12209 fs_manager.cc:359] Metadata directory not provided
I20260812 06:59:42.074221 12209 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:59:42.074455 12209 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 06:59:42.076632 12209 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786517687384214-19665-0/minicluster-data/master-0/data/instance:
uuid: "85018e2c1e7146dc807ac970e477e207"
format_stamp: "Formatted at 2026-08-12 06:59:42 on dist-test-slave-65mx"
server_key: "031fdf6994dd05152b1546007a3c1086"
server_key_iv: "968fb302fd197ec7ec8b291f8beee9bc"
server_key_version: "encryptionkey@0"
I20260812 06:59:42.077253 12209 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786517687384214-19665-0/minicluster-data/master-0/wal/instance:
uuid: "85018e2c1e7146dc807ac970e477e207"
format_stamp: "Formatted at 2026-08-12 06:59:42 on dist-test-slave-65mx"
server_key: "031fdf6994dd05152b1546007a3c1086"
server_key_iv: "968fb302fd197ec7ec8b291f8beee9bc"
server_key_version: "encryptionkey@0"
I20260812 06:59:42.081352 12209 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.001s
I20260812 06:59:42.084185 12226 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 06:59:42.085659 12209 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260812 06:59:42.085834 12209 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786517687384214-19665-0/minicluster-data/master-0/data,/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786517687384214-19665-0/minicluster-data/master-0/wal
uuid: "85018e2c1e7146dc807ac970e477e207"
format_stamp: "Formatted at 2026-08-12 06:59:42 on dist-test-slave-65mx"
server_key: "031fdf6994dd05152b1546007a3c1086"
server_key_iv: "968fb302fd197ec7ec8b291f8beee9bc"
server_key_version: "encryptionkey@0"
I20260812 06:59:42.086007 12209 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786517687384214-19665-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786517687384214-19665-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786517687384214-19665-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:59:42.090974 12209 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 06:59:42.094501 12209 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 06:59:42.094723 12209 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:59:42.102264 12209 rpc_server.cc:307] RPC server started. Bound to: 127.19.52.126:36055
I20260812 06:59:42.102296 12288 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.52.126:36055 every 8 connection(s)
I20260812 06:59:42.103273 12209 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786517687384214-19665-0/minicluster-data/master-0/data/info.pb
I20260812 06:59:42.106207 12289 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 06:59:42.111294 19665 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu as pid 12209
I20260812 06:59:42.111460 19665 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786517687384214-19665-0/minicluster-data/master-0/wal/instance
I20260812 06:59:42.111761 19665 external_mini_cluster.cc:1428] Setting key 2935f543bef72f3f013f6c2a50163aac
I20260812 06:59:42.112155 12289 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 85018e2c1e7146dc807ac970e477e207: Bootstrap starting.
I20260812 06:59:42.115463 12289 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 85018e2c1e7146dc807ac970e477e207: Neither blocks nor log segments found. Creating new log.
I20260812 06:59:42.116474 12289 log.cc:824] T 00000000000000000000000000000000 P 85018e2c1e7146dc807ac970e477e207: Log is configured to *not* fsync() on all Append() calls
I20260812 06:59:42.119616 12292 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:59:42.114205 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47369 (local address 127.19.52.126:36055)
0812 06:59:42.114829 (+ 624us) server_negotiation.cc:206] Beginning negotiation
0812 06:59:42.114849 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0812 06:59:42.114890 (+ 41us) server_negotiation.cc:407] Connection header received
0812 06:59:42.115836 (+ 946us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:59:42.115866 (+ 30us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:59:42.116238 (+ 372us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:59:42.117080 (+ 842us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:59:42.118110 (+ 1030us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:59:42.118118 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:59:42.118136 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 06:59:42.118149 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:59:42.118211 (+ 62us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 06:59:42.118243 (+ 32us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:59:42.118390 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0812 06:59:42.118557 (+ 167us) server_negotiation.cc:299] Negotiation successful
0812 06:59:42.118721 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":470,"thread_start_us":141,"threads_started":1}
I20260812 06:59:42.119706 12289 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 85018e2c1e7146dc807ac970e477e207: No bootstrap required, opened a new log
I20260812 06:59:42.122869 19665 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 06:59:42.124750 12289 raft_consensus.cc:348] T 00000000000000000000000000000000 P 85018e2c1e7146dc807ac970e477e207 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "85018e2c1e7146dc807ac970e477e207" member_type: VOTER last_known_addr { host: "127.19.52.126" port: 36055 } }
I20260812 06:59:42.125092 12289 raft_consensus.cc:374] T 00000000000000000000000000000000 P 85018e2c1e7146dc807ac970e477e207 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 06:59:42.125190 12289 raft_consensus.cc:729] T 00000000000000000000000000000000 P 85018e2c1e7146dc807ac970e477e207 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 85018e2c1e7146dc807ac970e477e207, State: Initialized, Role: FOLLOWER
I20260812 06:59:42.125739 12289 consensus_queue.cc:260] T 00000000000000000000000000000000 P 85018e2c1e7146dc807ac970e477e207 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "85018e2c1e7146dc807ac970e477e207" member_type: VOTER last_known_addr { host: "127.19.52.126" port: 36055 } }
I20260812 06:59:42.125929 12289 raft_consensus.cc:388] T 00000000000000000000000000000000 P 85018e2c1e7146dc807ac970e477e207 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 06:59:42.126020 12289 raft_consensus.cc:482] T 00000000000000000000000000000000 P 85018e2c1e7146dc807ac970e477e207 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 06:59:42.126155 12289 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 85018e2c1e7146dc807ac970e477e207 [term 0 FOLLOWER]: Advancing to term 1
I20260812 06:59:42.127758 12289 raft_consensus.cc:504] T 00000000000000000000000000000000 P 85018e2c1e7146dc807ac970e477e207 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "85018e2c1e7146dc807ac970e477e207" member_type: VOTER last_known_addr { host: "127.19.52.126" port: 36055 } }
I20260812 06:59:42.128193 12289 leader_election.cc:302] T 00000000000000000000000000000000 P 85018e2c1e7146dc807ac970e477e207 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 85018e2c1e7146dc807ac970e477e207; no voters:
I20260812 06:59:42.128633 12289 leader_election.cc:288] T 00000000000000000000000000000000 P 85018e2c1e7146dc807ac970e477e207 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 06:59:42.128784 12300 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 85018e2c1e7146dc807ac970e477e207 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 06:59:42.129134 12300 raft_consensus.cc:686] T 00000000000000000000000000000000 P 85018e2c1e7146dc807ac970e477e207 [term 1 LEADER]: Becoming Leader. State: Replica: 85018e2c1e7146dc807ac970e477e207, State: Running, Role: LEADER
I20260812 06:59:42.129490 12300 consensus_queue.cc:237] T 00000000000000000000000000000000 P 85018e2c1e7146dc807ac970e477e207 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "85018e2c1e7146dc807ac970e477e207" member_type: VOTER last_known_addr { host: "127.19.52.126" port: 36055 } }
I20260812 06:59:42.130286 12292 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:59:42.125498 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49831 (local address 127.19.52.126:36055)
0812 06:59:42.125708 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0812 06:59:42.125713 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 06:59:42.128362 (+ 2649us) server_negotiation.cc:407] Connection header received
0812 06:59:42.128628 (+ 266us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:59:42.128631 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:59:42.128690 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:59:42.128747 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:59:42.129828 (+ 1081us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:59:42.129831 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:59:42.129834 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 06:59:42.129848 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:59:42.129876 (+ 28us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 06:59:42.129889 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:59:42.129949 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0812 06:59:42.130124 (+ 175us) server_negotiation.cc:299] Negotiation successful
0812 06:59:42.130169 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":100}
I20260812 06:59:42.130574 12289 sys_catalog.cc:564] T 00000000000000000000000000000000 P 85018e2c1e7146dc807ac970e477e207 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 06:59:42.130950 12300 sys_catalog.cc:455] T 00000000000000000000000000000000 P 85018e2c1e7146dc807ac970e477e207 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 85018e2c1e7146dc807ac970e477e207. Latest consensus state: current_term: 1 leader_uuid: "85018e2c1e7146dc807ac970e477e207" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "85018e2c1e7146dc807ac970e477e207" member_type: VOTER last_known_addr { host: "127.19.52.126" port: 36055 } } }
I20260812 06:59:42.131078 12300 sys_catalog.cc:458] T 00000000000000000000000000000000 P 85018e2c1e7146dc807ac970e477e207 [sys.catalog]: This master's current role is: LEADER
I20260812 06:59:42.131392 12301 sys_catalog.cc:455] T 00000000000000000000000000000000 P 85018e2c1e7146dc807ac970e477e207 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "85018e2c1e7146dc807ac970e477e207" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "85018e2c1e7146dc807ac970e477e207" member_type: VOTER last_known_addr { host: "127.19.52.126" port: 36055 } } }
I20260812 06:59:42.131501 12301 sys_catalog.cc:458] T 00000000000000000000000000000000 P 85018e2c1e7146dc807ac970e477e207 [sys.catalog]: This master's current role is: LEADER
W20260812 06:59:42.135870 12317 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 85018e2c1e7146dc807ac970e477e207: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 06:59:42.135960 12317 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 06:59:42.136201 12309 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 06:59:42.137302 12309 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 06:59:42.143790 12309 catalog_manager.cc:1347] Generated new cluster ID: c4a54db90ade4adfb698457af9f45a3f
I20260812 06:59:42.143880 12309 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 06:59:42.183692 12309 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 06:59:42.184149 12309 catalog_manager.cc:1504] Loading token signing keys...
I20260812 06:59:42.190006 12309 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 85018e2c1e7146dc807ac970e477e207: Generated new TSK 0
I20260812 06:59:42.190703 12309 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 06:59:42.203465 19665 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu with pid 12209
2026-08-12T06:59:42Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/14 (1569 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/15
Loading random data
Initializing database '/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786517687384214-19665-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 06:59:42 dist-test-slave-65mx krb5kdc[12323](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 06:59:42 dist-test-slave-65mx krb5kdc[12323](info): set up 2 sockets
Aug 12 06:59:42 dist-test-slave-65mx krb5kdc[12323](info): commencing operation
krb5kdc: starting...
W20260812 06:59:43.347170 19665 mini_kdc.cc:121] Time spent starting KDC: real 1.120s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786517687384214-19665-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786517687384214-19665-0/krb5kdc/test-user.keytab.
Aug 12 06:59:43 dist-test-slave-65mx krb5kdc[12323](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517983, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T06:59:43Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T06:59:43Z Disabled control of system clock
WARNING: no policy specified for kudu/127.19.52.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.52.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.52.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786517687384214-19665-0/krb5kdc/kudu_127.19.52.126.keytab.
Entry for principal kudu/127.19.52.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786517687384214-19665-0/krb5kdc/kudu_127.19.52.126.keytab.
I20260812 06:59:43.496752 19665 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu
/tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786517687384214-19665-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786517687384214-19665-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786517687384214-19665-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786517687384214-19665-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.52.126:35783
--webserver_interface=127.19.52.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786517687384214-19665-0/krb5kdc/kudu_127.19.52.126.keytab
--principal=kudu/127.19.52.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.52.84:35051
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.19.52.126:35783
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786517687384214-19665-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786517687384214-19665-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786517687384214-19665-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 06:59:43.613478 12338 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:59:43.613818 12338 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:59:43.613904 12338 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 06:59:43.618045 12338 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 06:59:43.618163 12338 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 06:59:43.618212 12338 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 06:59:43.618268 12338 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 06:59:43.618309 12338 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 06:59:43.623816 12338 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.52.84:35051
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786517687384214-19665-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786517687384214-19665-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.19.52.126:35783
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786517687384214-19665-0/krb5kdc/kudu_127.19.52.126.keytab
--principal=kudu/127.19.52.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.52.126:35783
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786517687384214-19665-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.19.52.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12338
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786517687384214-19665-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 06:15:54 UTC on ccycloud.quasar-rnfzce.root.comops.site
build id 4581989
I20260812 06:59:43.625061 12338 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:59:43.626245 12338 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 06:59:43.635542 12348 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:59:43.635542 12345 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:59:43.635624 12346 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:59:43.635730 12338 server_base.cc:1034] running on GCE node
I20260812 06:59:43.636528 12338 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:59:43.637993 12338 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:59:43.639218 12338 hybrid_clock.cc:648] HybridClock initialized: now 1786517983639200 us; error 34 us; skew 500 ppm
Aug 12 06:59:43 dist-test-slave-65mx krb5kdc[12323](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517983, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.52.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 06:59:43.642481 12338 init.cc:377] Logged in from keytab as kudu/127.19.52.126@KRBTEST.COM (short username kudu)
I20260812 06:59:43.643522 12338 webserver.cc:460] Webserver started at http://127.19.52.126:38991/ using document root <none> and password file <none>
I20260812 06:59:43.644147 12338 fs_manager.cc:359] Metadata directory not provided
I20260812 06:59:43.644248 12338 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:59:43.644512 12338 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 06:59:43.646763 12338 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786517687384214-19665-0/minicluster-data/master-0/data/instance:
uuid: "df27bfd2831a4012a8bb6c10d1302d25"
format_stamp: "Formatted at 2026-08-12 06:59:43 on dist-test-slave-65mx"
server_key: "f7217f85f0d8dcd28620d4f8f168c51e"
server_key_iv: "e544e92735caab4c14b0901d660305c4"
server_key_version: "encryptionkey@0"
I20260812 06:59:43.647446 12338 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786517687384214-19665-0/minicluster-data/master-0/wal/instance:
uuid: "df27bfd2831a4012a8bb6c10d1302d25"
format_stamp: "Formatted at 2026-08-12 06:59:43 on dist-test-slave-65mx"
server_key: "f7217f85f0d8dcd28620d4f8f168c51e"
server_key_iv: "e544e92735caab4c14b0901d660305c4"
server_key_version: "encryptionkey@0"
I20260812 06:59:43.652074 12338 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.000s sys 0.006s
I20260812 06:59:43.655548 12356 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 06:59:43.657361 12338 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 06:59:43.657557 12338 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786517687384214-19665-0/minicluster-data/master-0/data,/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786517687384214-19665-0/minicluster-data/master-0/wal
uuid: "df27bfd2831a4012a8bb6c10d1302d25"
format_stamp: "Formatted at 2026-08-12 06:59:43 on dist-test-slave-65mx"
server_key: "f7217f85f0d8dcd28620d4f8f168c51e"
server_key_iv: "e544e92735caab4c14b0901d660305c4"
server_key_version: "encryptionkey@0"
I20260812 06:59:43.657713 12338 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786517687384214-19665-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786517687384214-19665-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786517687384214-19665-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:59:43.685735 12338 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 06:59:43.689218 12338 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 06:59:43.689440 12338 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:59:43.697800 12338 rpc_server.cc:307] RPC server started. Bound to: 127.19.52.126:35783
I20260812 06:59:43.697947 12414 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.52.126:35783 every 8 connection(s)
I20260812 06:59:43.698971 12338 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786517687384214-19665-0/minicluster-data/master-0/data/info.pb
I20260812 06:59:43.702149 12415 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 06:59:43.705953 19665 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu as pid 12338
I20260812 06:59:43.706125 19665 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786517687384214-19665-0/minicluster-data/master-0/wal/instance
I20260812 06:59:43.706517 19665 external_mini_cluster.cc:1428] Setting key dd0b55afdaf2f6f8ac0afed2db42ef34
I20260812 06:59:43.708964 12415 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P df27bfd2831a4012a8bb6c10d1302d25: Bootstrap starting.
I20260812 06:59:43.712061 12415 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P df27bfd2831a4012a8bb6c10d1302d25: Neither blocks nor log segments found. Creating new log.
I20260812 06:59:43.713137 12415 log.cc:824] T 00000000000000000000000000000000 P df27bfd2831a4012a8bb6c10d1302d25: Log is configured to *not* fsync() on all Append() calls
I20260812 06:59:43.715667 12415 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P df27bfd2831a4012a8bb6c10d1302d25: No bootstrap required, opened a new log
Aug 12 06:59:43 dist-test-slave-65mx krb5kdc[12323](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517983, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.19.52.126@KRBTEST.COM
I20260812 06:59:43.719796 12415 raft_consensus.cc:348] T 00000000000000000000000000000000 P df27bfd2831a4012a8bb6c10d1302d25 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "df27bfd2831a4012a8bb6c10d1302d25" member_type: VOTER last_known_addr { host: "127.19.52.126" port: 35783 } }
I20260812 06:59:43.720170 12415 raft_consensus.cc:374] T 00000000000000000000000000000000 P df27bfd2831a4012a8bb6c10d1302d25 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 06:59:43.720265 12415 raft_consensus.cc:729] T 00000000000000000000000000000000 P df27bfd2831a4012a8bb6c10d1302d25 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: df27bfd2831a4012a8bb6c10d1302d25, State: Initialized, Role: FOLLOWER
I20260812 06:59:43.720860 12415 consensus_queue.cc:260] T 00000000000000000000000000000000 P df27bfd2831a4012a8bb6c10d1302d25 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "df27bfd2831a4012a8bb6c10d1302d25" member_type: VOTER last_known_addr { host: "127.19.52.126" port: 35783 } }
I20260812 06:59:43.721035 12415 raft_consensus.cc:388] T 00000000000000000000000000000000 P df27bfd2831a4012a8bb6c10d1302d25 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 06:59:43.721138 12415 raft_consensus.cc:482] T 00000000000000000000000000000000 P df27bfd2831a4012a8bb6c10d1302d25 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 06:59:43.721259 12415 raft_consensus.cc:3037] T 00000000000000000000000000000000 P df27bfd2831a4012a8bb6c10d1302d25 [term 0 FOLLOWER]: Advancing to term 1
I20260812 06:59:43.722506 12415 raft_consensus.cc:504] T 00000000000000000000000000000000 P df27bfd2831a4012a8bb6c10d1302d25 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "df27bfd2831a4012a8bb6c10d1302d25" member_type: VOTER last_known_addr { host: "127.19.52.126" port: 35783 } }
I20260812 06:59:43.722678 12421 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:59:43.707952 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56465 (local address 127.19.52.126:35783)
0812 06:59:43.708403 (+ 451us) server_negotiation.cc:206] Beginning negotiation
0812 06:59:43.708419 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0812 06:59:43.708461 (+ 42us) server_negotiation.cc:407] Connection header received
0812 06:59:43.709218 (+ 757us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:59:43.709238 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:59:43.709585 (+ 347us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:59:43.709880 (+ 295us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:59:43.710901 (+ 1021us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:59:43.711937 (+ 1036us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:59:43.713278 (+ 1341us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:59:43.713875 (+ 597us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:59:43.714038 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:59:43.717113 (+ 3075us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:59:43.717119 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:59:43.717133 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:59:43.717162 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:59:43.719590 (+ 2428us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:59:43.720084 (+ 494us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:59:43.720089 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:59:43.720094 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:59:43.720249 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:59:43.720583 (+ 334us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:59:43.720589 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:59:43.720592 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:59:43.721037 (+ 445us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:59:43.721252 (+ 215us) server_negotiation.cc:1035] Waiting for connection context
0812 06:59:43.721543 (+ 291us) server_negotiation.cc:299] Negotiation successful
0812 06:59:43.721856 (+ 313us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":306,"thread_start_us":165,"threads_started":1}
I20260812 06:59:43.722947 12415 leader_election.cc:302] T 00000000000000000000000000000000 P df27bfd2831a4012a8bb6c10d1302d25 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: df27bfd2831a4012a8bb6c10d1302d25; no voters:
I20260812 06:59:43.723398 12415 leader_election.cc:288] T 00000000000000000000000000000000 P df27bfd2831a4012a8bb6c10d1302d25 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 06:59:43.723501 12424 raft_consensus.cc:2781] T 00000000000000000000000000000000 P df27bfd2831a4012a8bb6c10d1302d25 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 06:59:43.723765 12424 raft_consensus.cc:686] T 00000000000000000000000000000000 P df27bfd2831a4012a8bb6c10d1302d25 [term 1 LEADER]: Becoming Leader. State: Replica: df27bfd2831a4012a8bb6c10d1302d25, State: Running, Role: LEADER
I20260812 06:59:43.724206 12424 consensus_queue.cc:237] T 00000000000000000000000000000000 P df27bfd2831a4012a8bb6c10d1302d25 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "df27bfd2831a4012a8bb6c10d1302d25" member_type: VOTER last_known_addr { host: "127.19.52.126" port: 35783 } }
I20260812 06:59:43.724853 12415 sys_catalog.cc:564] T 00000000000000000000000000000000 P df27bfd2831a4012a8bb6c10d1302d25 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 06:59:43.726294 19665 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 06:59:43.726272 12426 sys_catalog.cc:455] T 00000000000000000000000000000000 P df27bfd2831a4012a8bb6c10d1302d25 [sys.catalog]: SysCatalogTable state changed. Reason: New leader df27bfd2831a4012a8bb6c10d1302d25. Latest consensus state: current_term: 1 leader_uuid: "df27bfd2831a4012a8bb6c10d1302d25" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "df27bfd2831a4012a8bb6c10d1302d25" member_type: VOTER last_known_addr { host: "127.19.52.126" port: 35783 } } }
I20260812 06:59:43.726402 12426 sys_catalog.cc:458] T 00000000000000000000000000000000 P df27bfd2831a4012a8bb6c10d1302d25 [sys.catalog]: This master's current role is: LEADER
I20260812 06:59:43.726820 12434 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 06:59:43.727370 12425 sys_catalog.cc:455] T 00000000000000000000000000000000 P df27bfd2831a4012a8bb6c10d1302d25 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "df27bfd2831a4012a8bb6c10d1302d25" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "df27bfd2831a4012a8bb6c10d1302d25" member_type: VOTER last_known_addr { host: "127.19.52.126" port: 35783 } } }
I20260812 06:59:43.727481 12425 sys_catalog.cc:458] T 00000000000000000000000000000000 P df27bfd2831a4012a8bb6c10d1302d25 [sys.catalog]: This master's current role is: LEADER
I20260812 06:59:43.730600 12434 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 06:59:43.736806 12434 catalog_manager.cc:1347] Generated new cluster ID: d5abcf9cd740450aabe9a3b53dd37efa
I20260812 06:59:43.736893 12434 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 06:59:43.742208 12421 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:59:43.729292 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43733 (local address 127.19.52.126:35783)
0812 06:59:43.729403 (+ 111us) server_negotiation.cc:206] Beginning negotiation
0812 06:59:43.729407 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 06:59:43.732875 (+ 3468us) server_negotiation.cc:407] Connection header received
0812 06:59:43.733030 (+ 155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:59:43.733034 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:59:43.733092 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:59:43.733195 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:59:43.734220 (+ 1025us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:59:43.734895 (+ 675us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:59:43.735958 (+ 1063us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:59:43.736342 (+ 384us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:59:43.736451 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:59:43.738482 (+ 2031us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:59:43.738486 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:59:43.738489 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:59:43.738517 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:59:43.740211 (+ 1694us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:59:43.740770 (+ 559us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:59:43.740774 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:59:43.740776 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:59:43.740873 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:59:43.741170 (+ 297us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:59:43.741173 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:59:43.741175 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:59:43.741413 (+ 238us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:59:43.741528 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0812 06:59:43.741899 (+ 371us) server_negotiation.cc:299] Negotiation successful
0812 06:59:43.742027 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":3,"server-negotiator.queue_time_us":42}
I20260812 06:59:43.759219 12434 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 06:59:43.760535 12434 catalog_manager.cc:1504] Loading token signing keys...
I20260812 06:59:43.775791 12434 catalog_manager.cc:5868] T 00000000000000000000000000000000 P df27bfd2831a4012a8bb6c10d1302d25: Generated new TSK 0
I20260812 06:59:43.776748 12434 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 06:59:43.783381 19665 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu with pid 12338
2026-08-12T06:59:43Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/15 (1580 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/16
Loading random data
Initializing database '/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786517687384214-19665-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 06:59:43 dist-test-slave-65mx krb5kdc[12453](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 06:59:43 dist-test-slave-65mx krb5kdc[12453](info): set up 2 sockets
Aug 12 06:59:43 dist-test-slave-65mx krb5kdc[12453](info): commencing operation
krb5kdc: starting...
W20260812 06:59:44.984637 19665 mini_kdc.cc:121] Time spent starting KDC: real 1.177s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786517687384214-19665-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786517687384214-19665-0/krb5kdc/test-user.keytab.
Aug 12 06:59:45 dist-test-slave-65mx krb5kdc[12453](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517985, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T06:59:45Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T06:59:45Z Disabled control of system clock
WARNING: no policy specified for kudu/127.19.52.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.52.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.52.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786517687384214-19665-0/krb5kdc/kudu_127.19.52.126.keytab.
Entry for principal kudu/127.19.52.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786517687384214-19665-0/krb5kdc/kudu_127.19.52.126.keytab.
I20260812 06:59:45.125138 19665 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu
/tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786517687384214-19665-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786517687384214-19665-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786517687384214-19665-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786517687384214-19665-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.52.126:44351
--webserver_interface=127.19.52.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786517687384214-19665-0/krb5kdc/kudu_127.19.52.126.keytab
--principal=kudu/127.19.52.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.52.84:46035
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.19.52.126:44351
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786517687384214-19665-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786517687384214-19665-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786517687384214-19665-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 06:59:45.241928 12468 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:59:45.242264 12468 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:59:45.242341 12468 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 06:59:45.246446 12468 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 06:59:45.246546 12468 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 06:59:45.246587 12468 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 06:59:45.246641 12468 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 06:59:45.246682 12468 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 06:59:45.252045 12468 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.52.84:46035
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786517687384214-19665-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786517687384214-19665-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.19.52.126:44351
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786517687384214-19665-0/krb5kdc/kudu_127.19.52.126.keytab
--principal=kudu/127.19.52.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.52.126:44351
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786517687384214-19665-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.19.52.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12468
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786517687384214-19665-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 06:15:54 UTC on ccycloud.quasar-rnfzce.root.comops.site
build id 4581989
I20260812 06:59:45.253191 12468 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:59:45.254244 12468 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 06:59:45.263265 12476 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:59:45.263249 12473 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:59:45.263249 12474 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:59:45.263367 12468 server_base.cc:1034] running on GCE node
I20260812 06:59:45.264223 12468 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:59:45.265482 12468 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:59:45.266671 12468 hybrid_clock.cc:648] HybridClock initialized: now 1786517985266634 us; error 47 us; skew 500 ppm
Aug 12 06:59:45 dist-test-slave-65mx krb5kdc[12453](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517985, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.52.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 06:59:45.269872 12468 init.cc:377] Logged in from keytab as kudu/127.19.52.126@KRBTEST.COM (short username kudu)
I20260812 06:59:45.270859 12468 webserver.cc:460] Webserver started at http://127.19.52.126:38607/ using document root <none> and password file <none>
I20260812 06:59:45.271471 12468 fs_manager.cc:359] Metadata directory not provided
I20260812 06:59:45.271567 12468 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:59:45.271806 12468 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 06:59:45.274118 12468 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786517687384214-19665-0/minicluster-data/master-0/data/instance:
uuid: "a85b71f237bd479db08d746d685c1fa6"
format_stamp: "Formatted at 2026-08-12 06:59:45 on dist-test-slave-65mx"
server_key: "f0f76035c4d559cd3a9104808f3a179f"
server_key_iv: "69c27ff961c147f26362cb704cb13bfe"
server_key_version: "encryptionkey@0"
I20260812 06:59:45.274727 12468 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786517687384214-19665-0/minicluster-data/master-0/wal/instance:
uuid: "a85b71f237bd479db08d746d685c1fa6"
format_stamp: "Formatted at 2026-08-12 06:59:45 on dist-test-slave-65mx"
server_key: "f0f76035c4d559cd3a9104808f3a179f"
server_key_iv: "69c27ff961c147f26362cb704cb13bfe"
server_key_version: "encryptionkey@0"
I20260812 06:59:45.279225 12468 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260812 06:59:45.282038 12483 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 06:59:45.283556 12468 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260812 06:59:45.283730 12468 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786517687384214-19665-0/minicluster-data/master-0/data,/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786517687384214-19665-0/minicluster-data/master-0/wal
uuid: "a85b71f237bd479db08d746d685c1fa6"
format_stamp: "Formatted at 2026-08-12 06:59:45 on dist-test-slave-65mx"
server_key: "f0f76035c4d559cd3a9104808f3a179f"
server_key_iv: "69c27ff961c147f26362cb704cb13bfe"
server_key_version: "encryptionkey@0"
I20260812 06:59:45.283864 12468 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786517687384214-19665-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786517687384214-19665-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786517687384214-19665-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:59:45.306741 12468 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 06:59:45.310165 12468 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 06:59:45.310379 12468 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:59:45.317798 12468 rpc_server.cc:307] RPC server started. Bound to: 127.19.52.126:44351
I20260812 06:59:45.317828 12548 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.52.126:44351 every 8 connection(s)
I20260812 06:59:45.318789 12468 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786517687384214-19665-0/minicluster-data/master-0/data/info.pb
I20260812 06:59:45.321712 12549 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 06:59:45.324088 19665 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu as pid 12468
I20260812 06:59:45.324209 19665 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786517687384214-19665-0/minicluster-data/master-0/wal/instance
I20260812 06:59:45.324532 19665 external_mini_cluster.cc:1428] Setting key dadd4a1feeff73e710bb2eaaa5103db5
I20260812 06:59:45.328485 12549 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a85b71f237bd479db08d746d685c1fa6: Bootstrap starting.
I20260812 06:59:45.331044 12549 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a85b71f237bd479db08d746d685c1fa6: Neither blocks nor log segments found. Creating new log.
I20260812 06:59:45.332264 12549 log.cc:824] T 00000000000000000000000000000000 P a85b71f237bd479db08d746d685c1fa6: Log is configured to *not* fsync() on all Append() calls
I20260812 06:59:45.334448 12552 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:59:45.326111 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42563 (local address 127.19.52.126:44351)
0812 06:59:45.326613 (+ 502us) server_negotiation.cc:206] Beginning negotiation
0812 06:59:45.326629 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0812 06:59:45.326668 (+ 39us) server_negotiation.cc:407] Connection header received
0812 06:59:45.327476 (+ 808us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:59:45.327497 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:59:45.327861 (+ 364us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:59:45.328213 (+ 352us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:59:45.328750 (+ 537us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:59:45.329756 (+ 1006us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:59:45.330700 (+ 944us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:59:45.331241 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:59:45.331417 (+ 176us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:59:45.332529 (+ 1112us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:59:45.332575 (+ 46us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:59:45.332601 (+ 26us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 06:59:45.333039 (+ 438us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:59:45.333101 (+ 62us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 06:59:45.333125 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:59:45.333243 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0812 06:59:45.333414 (+ 171us) server_negotiation.cc:299] Negotiation successful
0812 06:59:45.333665 (+ 251us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":342,"thread_start_us":166,"threads_started":1}
I20260812 06:59:45.334970 12549 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a85b71f237bd479db08d746d685c1fa6: No bootstrap required, opened a new log
I20260812 06:59:45.337481 19665 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 06:59:45.339979 12549 raft_consensus.cc:348] T 00000000000000000000000000000000 P a85b71f237bd479db08d746d685c1fa6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a85b71f237bd479db08d746d685c1fa6" member_type: VOTER last_known_addr { host: "127.19.52.126" port: 44351 } }
I20260812 06:59:45.340363 12549 raft_consensus.cc:374] T 00000000000000000000000000000000 P a85b71f237bd479db08d746d685c1fa6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 06:59:45.340443 12549 raft_consensus.cc:729] T 00000000000000000000000000000000 P a85b71f237bd479db08d746d685c1fa6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a85b71f237bd479db08d746d685c1fa6, State: Initialized, Role: FOLLOWER
I20260812 06:59:45.341004 12549 consensus_queue.cc:260] T 00000000000000000000000000000000 P a85b71f237bd479db08d746d685c1fa6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a85b71f237bd479db08d746d685c1fa6" member_type: VOTER last_known_addr { host: "127.19.52.126" port: 44351 } }
I20260812 06:59:45.341159 12549 raft_consensus.cc:388] T 00000000000000000000000000000000 P a85b71f237bd479db08d746d685c1fa6 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 06:59:45.341253 12549 raft_consensus.cc:482] T 00000000000000000000000000000000 P a85b71f237bd479db08d746d685c1fa6 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 06:59:45.341352 12549 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a85b71f237bd479db08d746d685c1fa6 [term 0 FOLLOWER]: Advancing to term 1
I20260812 06:59:45.342698 12549 raft_consensus.cc:504] T 00000000000000000000000000000000 P a85b71f237bd479db08d746d685c1fa6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a85b71f237bd479db08d746d685c1fa6" member_type: VOTER last_known_addr { host: "127.19.52.126" port: 44351 } }
I20260812 06:59:45.343081 12549 leader_election.cc:302] T 00000000000000000000000000000000 P a85b71f237bd479db08d746d685c1fa6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a85b71f237bd479db08d746d685c1fa6; no voters:
I20260812 06:59:45.343441 12549 leader_election.cc:288] T 00000000000000000000000000000000 P a85b71f237bd479db08d746d685c1fa6 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 06:59:45.343560 12560 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a85b71f237bd479db08d746d685c1fa6 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 06:59:45.343819 12560 raft_consensus.cc:686] T 00000000000000000000000000000000 P a85b71f237bd479db08d746d685c1fa6 [term 1 LEADER]: Becoming Leader. State: Replica: a85b71f237bd479db08d746d685c1fa6, State: Running, Role: LEADER
I20260812 06:59:45.344213 12560 consensus_queue.cc:237] T 00000000000000000000000000000000 P a85b71f237bd479db08d746d685c1fa6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a85b71f237bd479db08d746d685c1fa6" member_type: VOTER last_known_addr { host: "127.19.52.126" port: 44351 } }
I20260812 06:59:45.344832 12552 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:59:45.339995 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33793 (local address 127.19.52.126:44351)
0812 06:59:45.340140 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0812 06:59:45.340154 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0812 06:59:45.340379 (+ 225us) server_negotiation.cc:407] Connection header received
0812 06:59:45.340568 (+ 189us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:59:45.340572 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:59:45.340627 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:59:45.340705 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:59:45.341009 (+ 304us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:59:45.341704 (+ 695us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:59:45.342868 (+ 1164us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:59:45.343121 (+ 253us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:59:45.343246 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:59:45.344179 (+ 933us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:59:45.344183 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:59:45.344185 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 06:59:45.344200 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:59:45.344231 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 06:59:45.344244 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:59:45.344309 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0812 06:59:45.344652 (+ 343us) server_negotiation.cc:299] Negotiation successful
0812 06:59:45.344711 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260812 06:59:45.345007 12549 sys_catalog.cc:564] T 00000000000000000000000000000000 P a85b71f237bd479db08d746d685c1fa6 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 06:59:45.345985 12562 sys_catalog.cc:455] T 00000000000000000000000000000000 P a85b71f237bd479db08d746d685c1fa6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a85b71f237bd479db08d746d685c1fa6. Latest consensus state: current_term: 1 leader_uuid: "a85b71f237bd479db08d746d685c1fa6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a85b71f237bd479db08d746d685c1fa6" member_type: VOTER last_known_addr { host: "127.19.52.126" port: 44351 } } }
I20260812 06:59:45.346139 12562 sys_catalog.cc:458] T 00000000000000000000000000000000 P a85b71f237bd479db08d746d685c1fa6 [sys.catalog]: This master's current role is: LEADER
I20260812 06:59:45.346153 12561 sys_catalog.cc:455] T 00000000000000000000000000000000 P a85b71f237bd479db08d746d685c1fa6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a85b71f237bd479db08d746d685c1fa6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a85b71f237bd479db08d746d685c1fa6" member_type: VOTER last_known_addr { host: "127.19.52.126" port: 44351 } } }
I20260812 06:59:45.346506 12561 sys_catalog.cc:458] T 00000000000000000000000000000000 P a85b71f237bd479db08d746d685c1fa6 [sys.catalog]: This master's current role is: LEADER
I20260812 06:59:45.346572 12569 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 06:59:45.350288 12569 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 06:59:45.356281 12569 catalog_manager.cc:1347] Generated new cluster ID: 690c8cd199bd42698fa5a4d1b22e886d
I20260812 06:59:45.356365 12569 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 06:59:45.374678 12569 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 06:59:45.375900 12569 catalog_manager.cc:1504] Loading token signing keys...
I20260812 06:59:45.385504 12569 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a85b71f237bd479db08d746d685c1fa6: Generated new TSK 0
I20260812 06:59:45.386263 12569 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 06:59:45.396386 19665 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu with pid 12468
2026-08-12T06:59:45Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/16 (1611 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/17
Loading random data
Initializing database '/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786517687384214-19665-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 06:59:45 dist-test-slave-65mx krb5kdc[12583](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 06:59:45 dist-test-slave-65mx krb5kdc[12583](info): set up 2 sockets
Aug 12 06:59:45 dist-test-slave-65mx krb5kdc[12583](info): commencing operation
krb5kdc: starting...
W20260812 06:59:46.536028 19665 mini_kdc.cc:121] Time spent starting KDC: real 1.117s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786517687384214-19665-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786517687384214-19665-0/krb5kdc/test-user.keytab.
Aug 12 06:59:46 dist-test-slave-65mx krb5kdc[12583](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517986, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T06:59:46Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T06:59:46Z Disabled control of system clock
WARNING: no policy specified for kudu/127.19.52.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.52.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.52.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786517687384214-19665-0/krb5kdc/kudu_127.19.52.126.keytab.
Entry for principal kudu/127.19.52.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786517687384214-19665-0/krb5kdc/kudu_127.19.52.126.keytab.
I20260812 06:59:46.678299 19665 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu
/tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786517687384214-19665-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786517687384214-19665-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786517687384214-19665-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786517687384214-19665-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.52.126:37963
--webserver_interface=127.19.52.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786517687384214-19665-0/krb5kdc/kudu_127.19.52.126.keytab
--principal=kudu/127.19.52.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.52.84:39287
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.19.52.126:37963
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786517687384214-19665-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786517687384214-19665-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786517687384214-19665-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 06:59:46.793517 12599 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:59:46.793825 12599 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:59:46.793872 12599 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 06:59:46.798013 12599 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 06:59:46.798084 12599 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 06:59:46.798105 12599 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 06:59:46.798121 12599 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 06:59:46.798137 12599 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 06:59:46.803881 12599 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.52.84:39287
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786517687384214-19665-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786517687384214-19665-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.19.52.126:37963
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786517687384214-19665-0/krb5kdc/kudu_127.19.52.126.keytab
--principal=kudu/127.19.52.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.52.126:37963
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786517687384214-19665-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.19.52.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12599
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786517687384214-19665-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 06:15:54 UTC on ccycloud.quasar-rnfzce.root.comops.site
build id 4581989
I20260812 06:59:46.805094 12599 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:59:46.806162 12599 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 06:59:46.815505 12609 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:59:46.815505 12606 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:59:46.815642 12607 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:59:46.815660 12599 server_base.cc:1034] running on GCE node
I20260812 06:59:46.816705 12599 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:59:46.818112 12599 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:59:46.819314 12599 hybrid_clock.cc:648] HybridClock initialized: now 1786517986819263 us; error 58 us; skew 500 ppm
Aug 12 06:59:46 dist-test-slave-65mx krb5kdc[12583](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517986, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.52.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 06:59:46.822568 12599 init.cc:377] Logged in from keytab as kudu/127.19.52.126@KRBTEST.COM (short username kudu)
I20260812 06:59:46.823617 12599 webserver.cc:460] Webserver started at http://127.19.52.126:37775/ using document root <none> and password file <none>
I20260812 06:59:46.824254 12599 fs_manager.cc:359] Metadata directory not provided
I20260812 06:59:46.824348 12599 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:59:46.824638 12599 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 06:59:46.826830 12599 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786517687384214-19665-0/minicluster-data/master-0/data/instance:
uuid: "06da6853d82f4c55ac21158ed1f789e8"
format_stamp: "Formatted at 2026-08-12 06:59:46 on dist-test-slave-65mx"
server_key: "19dbf38082f6398e2782425c6d311aaa"
server_key_iv: "f30837d9895359c822789f4ecf78aeb6"
server_key_version: "encryptionkey@0"
I20260812 06:59:46.827428 12599 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786517687384214-19665-0/minicluster-data/master-0/wal/instance:
uuid: "06da6853d82f4c55ac21158ed1f789e8"
format_stamp: "Formatted at 2026-08-12 06:59:46 on dist-test-slave-65mx"
server_key: "19dbf38082f6398e2782425c6d311aaa"
server_key_iv: "f30837d9895359c822789f4ecf78aeb6"
server_key_version: "encryptionkey@0"
I20260812 06:59:46.831626 12599 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.004s
I20260812 06:59:46.834507 12617 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 06:59:46.836015 12599 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260812 06:59:46.836201 12599 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786517687384214-19665-0/minicluster-data/master-0/data,/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786517687384214-19665-0/minicluster-data/master-0/wal
uuid: "06da6853d82f4c55ac21158ed1f789e8"
format_stamp: "Formatted at 2026-08-12 06:59:46 on dist-test-slave-65mx"
server_key: "19dbf38082f6398e2782425c6d311aaa"
server_key_iv: "f30837d9895359c822789f4ecf78aeb6"
server_key_version: "encryptionkey@0"
I20260812 06:59:46.836341 12599 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786517687384214-19665-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786517687384214-19665-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786517687384214-19665-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:59:46.841455 12599 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 06:59:46.844974 12599 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 06:59:46.845194 12599 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:59:46.853259 12599 rpc_server.cc:307] RPC server started. Bound to: 127.19.52.126:37963
I20260812 06:59:46.853291 12675 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.52.126:37963 every 8 connection(s)
I20260812 06:59:46.854236 12599 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786517687384214-19665-0/minicluster-data/master-0/data/info.pb
I20260812 06:59:46.856405 19665 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu as pid 12599
I20260812 06:59:46.856617 19665 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786517687384214-19665-0/minicluster-data/master-0/wal/instance
I20260812 06:59:46.856968 19665 external_mini_cluster.cc:1428] Setting key 33f1d9aaa8dc13a40da86876471b3080
I20260812 06:59:46.857662 12676 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 06:59:46.863229 12679 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:59:46.858535 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41041 (local address 127.19.52.126:37963)
0812 06:59:46.859018 (+ 483us) server_negotiation.cc:206] Beginning negotiation
0812 06:59:46.859041 (+ 23us) server_negotiation.cc:399] Waiting for connection header
0812 06:59:46.859088 (+ 47us) server_negotiation.cc:407] Connection header received
0812 06:59:46.860018 (+ 930us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:59:46.860043 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:59:46.860440 (+ 397us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:59:46.860896 (+ 456us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:59:46.861656 (+ 760us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:59:46.861662 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:59:46.861678 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 06:59:46.861692 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:59:46.861759 (+ 67us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 06:59:46.861789 (+ 30us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:59:46.861951 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0812 06:59:46.862032 (+ 81us) server_negotiation.cc:299] Negotiation successful
0812 06:59:46.862198 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":318,"thread_start_us":141,"threads_started":1}
I20260812 06:59:46.865895 12676 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 06da6853d82f4c55ac21158ed1f789e8: Bootstrap starting.
I20260812 06:59:46.866605 19665 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 06:59:46.868396 12676 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 06da6853d82f4c55ac21158ed1f789e8: Neither blocks nor log segments found. Creating new log.
I20260812 06:59:46.869416 12676 log.cc:824] T 00000000000000000000000000000000 P 06da6853d82f4c55ac21158ed1f789e8: Log is configured to *not* fsync() on all Append() calls
I20260812 06:59:46.870428 12679 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:59:46.868702 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41315 (local address 127.19.52.126:37963)
0812 06:59:46.868847 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0812 06:59:46.868854 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 06:59:46.869028 (+ 174us) server_negotiation.cc:407] Connection header received
0812 06:59:46.869147 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:59:46.869150 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:59:46.869183 (+ 33us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:59:46.869253 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:59:46.870012 (+ 759us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:59:46.870016 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:59:46.870019 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 06:59:46.870030 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:59:46.870056 (+ 26us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 06:59:46.870072 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:59:46.870162 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0812 06:59:46.870266 (+ 104us) server_negotiation.cc:299] Negotiation successful
0812 06:59:46.870303 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260812 06:59:46.871860 12676 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 06da6853d82f4c55ac21158ed1f789e8: No bootstrap required, opened a new log
I20260812 06:59:46.875883 12676 raft_consensus.cc:348] T 00000000000000000000000000000000 P 06da6853d82f4c55ac21158ed1f789e8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "06da6853d82f4c55ac21158ed1f789e8" member_type: VOTER last_known_addr { host: "127.19.52.126" port: 37963 } }
I20260812 06:59:46.876150 12676 raft_consensus.cc:374] T 00000000000000000000000000000000 P 06da6853d82f4c55ac21158ed1f789e8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 06:59:46.876188 12676 raft_consensus.cc:729] T 00000000000000000000000000000000 P 06da6853d82f4c55ac21158ed1f789e8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 06da6853d82f4c55ac21158ed1f789e8, State: Initialized, Role: FOLLOWER
I20260812 06:59:46.876716 12676 consensus_queue.cc:260] T 00000000000000000000000000000000 P 06da6853d82f4c55ac21158ed1f789e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "06da6853d82f4c55ac21158ed1f789e8" member_type: VOTER last_known_addr { host: "127.19.52.126" port: 37963 } }
I20260812 06:59:46.876895 12676 raft_consensus.cc:388] T 00000000000000000000000000000000 P 06da6853d82f4c55ac21158ed1f789e8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 06:59:46.876994 12676 raft_consensus.cc:482] T 00000000000000000000000000000000 P 06da6853d82f4c55ac21158ed1f789e8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 06:59:46.877112 12676 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 06da6853d82f4c55ac21158ed1f789e8 [term 0 FOLLOWER]: Advancing to term 1
I20260812 06:59:46.878293 12676 raft_consensus.cc:504] T 00000000000000000000000000000000 P 06da6853d82f4c55ac21158ed1f789e8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "06da6853d82f4c55ac21158ed1f789e8" member_type: VOTER last_known_addr { host: "127.19.52.126" port: 37963 } }
I20260812 06:59:46.878659 12676 leader_election.cc:302] T 00000000000000000000000000000000 P 06da6853d82f4c55ac21158ed1f789e8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 06da6853d82f4c55ac21158ed1f789e8; no voters:
I20260812 06:59:46.879052 12676 leader_election.cc:288] T 00000000000000000000000000000000 P 06da6853d82f4c55ac21158ed1f789e8 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 06:59:46.879169 12686 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 06da6853d82f4c55ac21158ed1f789e8 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 06:59:46.879431 12686 raft_consensus.cc:686] T 00000000000000000000000000000000 P 06da6853d82f4c55ac21158ed1f789e8 [term 1 LEADER]: Becoming Leader. State: Replica: 06da6853d82f4c55ac21158ed1f789e8, State: Running, Role: LEADER
I20260812 06:59:46.879788 12686 consensus_queue.cc:237] T 00000000000000000000000000000000 P 06da6853d82f4c55ac21158ed1f789e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "06da6853d82f4c55ac21158ed1f789e8" member_type: VOTER last_known_addr { host: "127.19.52.126" port: 37963 } }
I20260812 06:59:46.880369 12676 sys_catalog.cc:564] T 00000000000000000000000000000000 P 06da6853d82f4c55ac21158ed1f789e8 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 06:59:46.881567 12687 sys_catalog.cc:455] T 00000000000000000000000000000000 P 06da6853d82f4c55ac21158ed1f789e8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "06da6853d82f4c55ac21158ed1f789e8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "06da6853d82f4c55ac21158ed1f789e8" member_type: VOTER last_known_addr { host: "127.19.52.126" port: 37963 } } }
I20260812 06:59:46.881706 12687 sys_catalog.cc:458] T 00000000000000000000000000000000 P 06da6853d82f4c55ac21158ed1f789e8 [sys.catalog]: This master's current role is: LEADER
I20260812 06:59:46.881860 12688 sys_catalog.cc:455] T 00000000000000000000000000000000 P 06da6853d82f4c55ac21158ed1f789e8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 06da6853d82f4c55ac21158ed1f789e8. Latest consensus state: current_term: 1 leader_uuid: "06da6853d82f4c55ac21158ed1f789e8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "06da6853d82f4c55ac21158ed1f789e8" member_type: VOTER last_known_addr { host: "127.19.52.126" port: 37963 } } }
I20260812 06:59:46.881958 12688 sys_catalog.cc:458] T 00000000000000000000000000000000 P 06da6853d82f4c55ac21158ed1f789e8 [sys.catalog]: This master's current role is: LEADER
I20260812 06:59:46.882220 12695 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 06:59:46.885452 12695 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 06:59:46.891233 12695 catalog_manager.cc:1347] Generated new cluster ID: 1be61d660dad4023abc3421ebf0d69e3
I20260812 06:59:46.891405 12695 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 06:59:46.906662 12695 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 06:59:46.907028 12695 catalog_manager.cc:1504] Loading token signing keys...
I20260812 06:59:46.913328 12695 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 06da6853d82f4c55ac21158ed1f789e8: Generated new TSK 0
I20260812 06:59:46.914036 12695 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 06:59:46.918749 19665 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu with pid 12599
2026-08-12T06:59:46Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/17 (1523 ms)
[----------] 18 tests from AuthTokenIssuingTest (28988 ms total)
[----------] 2 tests from ConnectToFollowerMasterTest
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0
Loading random data
Initializing database '/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786517687384214-19665-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 06:59:46 dist-test-slave-65mx krb5kdc[12709](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 06:59:46 dist-test-slave-65mx krb5kdc[12709](info): set up 2 sockets
Aug 12 06:59:46 dist-test-slave-65mx krb5kdc[12709](info): commencing operation
krb5kdc: starting...
W20260812 06:59:48.074654 19665 mini_kdc.cc:121] Time spent starting KDC: real 1.132s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786517687384214-19665-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786517687384214-19665-0/krb5kdc/test-user.keytab.
Aug 12 06:59:48 dist-test-slave-65mx krb5kdc[12709](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517988, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T06:59:48Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T06:59:48Z Disabled control of system clock
WARNING: no policy specified for kudu/127.19.52.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.52.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.52.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786517687384214-19665-0/krb5kdc/kudu_127.19.52.126.keytab.
Entry for principal kudu/127.19.52.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786517687384214-19665-0/krb5kdc/kudu_127.19.52.126.keytab.
I20260812 06:59:48.216404 19665 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu
/tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786517687384214-19665-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786517687384214-19665-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786517687384214-19665-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786517687384214-19665-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.52.126:42867
--webserver_interface=127.19.52.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786517687384214-19665-0/krb5kdc/kudu_127.19.52.126.keytab
--principal=kudu/127.19.52.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.52.84:45431
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.19.52.126:42867,127.19.52.125:42015,127.19.52.124:33027
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786517687384214-19665-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786517687384214-19665-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786517687384214-19665-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 06:59:48.334400 12724 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260812 06:59:48.334815 12724 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:59:48.334894 12724 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:59:48.334980 12724 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 06:59:48.339512 12724 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 06:59:48.339643 12724 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 06:59:48.339684 12724 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 06:59:48.339725 12724 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 06:59:48.339761 12724 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 06:59:48.345605 12724 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.52.84:45431
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786517687384214-19665-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786517687384214-19665-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.19.52.126:42867,127.19.52.125:42015,127.19.52.124:33027
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786517687384214-19665-0/krb5kdc/kudu_127.19.52.126.keytab
--principal=kudu/127.19.52.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.52.126:42867
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786517687384214-19665-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.19.52.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12724
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786517687384214-19665-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 06:15:54 UTC on ccycloud.quasar-rnfzce.root.comops.site
build id 4581989
I20260812 06:59:48.346824 12724 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:59:48.347900 12724 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 06:59:48.357868 12732 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:59:48.357879 12729 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:59:48.357928 12724 server_base.cc:1034] running on GCE node
W20260812 06:59:48.358040 12730 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:59:48.358783 12724 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:59:48.359979 12724 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:59:48.361204 12724 hybrid_clock.cc:648] HybridClock initialized: now 1786517988361187 us; error 26 us; skew 500 ppm
Aug 12 06:59:48 dist-test-slave-65mx krb5kdc[12709](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517988, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.52.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 06:59:48.369576 12724 init.cc:377] Logged in from keytab as kudu/127.19.52.126@KRBTEST.COM (short username kudu)
I20260812 06:59:48.370963 12724 webserver.cc:460] Webserver started at http://127.19.52.126:33299/ using document root <none> and password file <none>
I20260812 06:59:48.371654 12724 fs_manager.cc:359] Metadata directory not provided
I20260812 06:59:48.371760 12724 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:59:48.372021 12724 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 06:59:48.374403 12724 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786517687384214-19665-0/minicluster-data/master-0/data/instance:
uuid: "1f70bae80ad9489b91ebf7370ecdd4d9"
format_stamp: "Formatted at 2026-08-12 06:59:48 on dist-test-slave-65mx"
server_key: "813e9e565f316e3976decf98418620c4"
server_key_iv: "026c710ae8b329d835b461c0ab5c7137"
server_key_version: "encryptionkey@0"
I20260812 06:59:48.375262 12724 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786517687384214-19665-0/minicluster-data/master-0/wal/instance:
uuid: "1f70bae80ad9489b91ebf7370ecdd4d9"
format_stamp: "Formatted at 2026-08-12 06:59:48 on dist-test-slave-65mx"
server_key: "813e9e565f316e3976decf98418620c4"
server_key_iv: "026c710ae8b329d835b461c0ab5c7137"
server_key_version: "encryptionkey@0"
I20260812 06:59:48.379689 12724 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260812 06:59:48.382679 12741 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 06:59:48.384692 12724 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 06:59:48.384881 12724 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786517687384214-19665-0/minicluster-data/master-0/data,/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786517687384214-19665-0/minicluster-data/master-0/wal
uuid: "1f70bae80ad9489b91ebf7370ecdd4d9"
format_stamp: "Formatted at 2026-08-12 06:59:48 on dist-test-slave-65mx"
server_key: "813e9e565f316e3976decf98418620c4"
server_key_iv: "026c710ae8b329d835b461c0ab5c7137"
server_key_version: "encryptionkey@0"
I20260812 06:59:48.385027 12724 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786517687384214-19665-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786517687384214-19665-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786517687384214-19665-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:59:48.410359 12724 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 06:59:48.413650 12724 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 06:59:48.413817 12724 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:59:48.420691 12724 rpc_server.cc:307] RPC server started. Bound to: 127.19.52.126:42867
I20260812 06:59:48.421007 12801 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.52.126:42867 every 8 connection(s)
I20260812 06:59:48.421715 12724 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786517687384214-19665-0/minicluster-data/master-0/data/info.pb
I20260812 06:59:48.424619 12802 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 06:59:48.424889 19665 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu as pid 12724
I20260812 06:59:48.425033 19665 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786517687384214-19665-0/minicluster-data/master-0/wal/instance
I20260812 06:59:48.425339 19665 external_mini_cluster.cc:1428] Setting key ab14b47c751b44135cf4e5b26bac0aee
I20260812 06:59:48.429917 12802 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.19.52.126" port: 42867 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.19.52.125@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.52.125@KRBTEST.COM" created.
Aug 12 06:59:48 dist-test-slave-65mx krb5kdc[12709](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517988, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.52.126@KRBTEST.COM for kudu/127.19.52.126@KRBTEST.COM
I20260812 06:59:48.447901 12805 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:59:48.433169 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.126:42867 (local address 127.0.0.1:35348)
0812 06:59:48.433732 (+ 563us) negotiation.cc:107] Waiting for socket to connect
0812 06:59:48.433788 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0812 06:59:48.434729 (+ 941us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:59:48.435772 (+ 1043us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:59:48.435786 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:59:48.436236 (+ 450us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:59:48.436523 (+ 287us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:59:48.436603 (+ 80us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:59:48.438014 (+ 1411us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:59:48.438019 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:59:48.439021 (+ 1002us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:59:48.439032 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:59:48.440002 (+ 970us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:59:48.440006 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:59:48.440102 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:59:48.440842 (+ 740us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 06:59:48.440859 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:59:48.442630 (+ 1771us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 06:59:48.444749 (+ 2119us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:59:48.444757 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:59:48.444768 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:59:48.445003 (+ 235us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:59:48.445351 (+ 348us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:59:48.445355 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:59:48.445357 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:59:48.445483 (+ 126us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:59:48.446119 (+ 636us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 06:59:48.446125 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 06:59:48.446268 (+ 143us) client_negotiation.cc:769] Sending connection context
0812 06:59:48.446433 (+ 165us) client_negotiation.cc:240] Negotiation successful
0812 06:59:48.446830 (+ 397us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":349,"mutex_wait_us":107,"spinlock_wait_cycles":5504,"thread_start_us":172,"threads_started":1}
I20260812 06:59:48.447901 12807 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:59:48.433170 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35348 (local address 127.19.52.126:42867)
0812 06:59:48.433841 (+ 671us) server_negotiation.cc:206] Beginning negotiation
0812 06:59:48.433854 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0812 06:59:48.433901 (+ 47us) server_negotiation.cc:407] Connection header received
0812 06:59:48.435098 (+ 1197us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:59:48.435106 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:59:48.435342 (+ 236us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:59:48.435518 (+ 176us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:59:48.436767 (+ 1249us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:59:48.437781 (+ 1014us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:59:48.439151 (+ 1370us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:59:48.439790 (+ 639us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:59:48.439989 (+ 199us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:59:48.442805 (+ 2816us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:59:48.442811 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:59:48.442822 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:59:48.442844 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:59:48.444631 (+ 1787us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:59:48.445122 (+ 491us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:59:48.445132 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:59:48.445134 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:59:48.445257 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:59:48.445581 (+ 324us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:59:48.445590 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:59:48.445592 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:59:48.445997 (+ 405us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:59:48.446138 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0812 06:59:48.446513 (+ 375us) server_negotiation.cc:299] Negotiation successful
0812 06:59:48.446769 (+ 256us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":546,"spinlock_wait_cycles":34688,"thread_start_us":220,"threads_started":1}
I20260812 06:59:48.450768 12802 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.19.52.125" port: 42015 } has no permanent_uuid. Determining permanent_uuid...
I20260812 06:59:48.451719 12805 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:59:48.451362 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.125:42015 (local address 127.0.0.1:60638)
0812 06:59:48.451533 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0812 06:59:48.451645 (+ 112us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.52.125:42015: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
W20260812 06:59:48.451962 12745 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.19.52.125:42015: connect: Connection refused (error 111)
I20260812 06:59:48.453066 12805 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:59:48.452828 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.125:42015 (local address 127.0.0.1:60654)
0812 06:59:48.452941 (+ 113us) negotiation.cc:107] Waiting for socket to connect
0812 06:59:48.453008 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.52.125:42015: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
W20260812 06:59:48.453239 12802 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.19.52.125:42015: Network error: Client connection negotiation failed: client connection to 127.19.52.125:42015: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.52.125 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786517687384214-19665-0/krb5kdc/kudu_127.19.52.125.keytab.
Entry for principal kudu/127.19.52.125 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786517687384214-19665-0/krb5kdc/kudu_127.19.52.125.keytab.
I20260812 06:59:48.456338 19665 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu
/tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786517687384214-19665-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786517687384214-19665-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786517687384214-19665-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786517687384214-19665-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.52.125:42015
--webserver_interface=127.19.52.125
--webserver_port=0
--keytab_file=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786517687384214-19665-0/krb5kdc/kudu_127.19.52.125.keytab
--principal=kudu/127.19.52.125
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.52.84:45431
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.19.52.126:42867,127.19.52.125:42015,127.19.52.124:33027
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786517687384214-19665-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786517687384214-19665-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786517687384214-19665-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 06:59:48.502561 12802 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.19.52.125" port: 42015 } attempt: 1
I20260812 06:59:48.503541 12805 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:59:48.503197 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.125:42015 (local address 127.0.0.1:60660)
0812 06:59:48.503366 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0812 06:59:48.503458 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.52.125:42015: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":73}
I20260812 06:59:48.504489 12805 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:59:48.504231 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.125:42015 (local address 127.0.0.1:60674)
0812 06:59:48.504357 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0812 06:59:48.504408 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.52.125:42015: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
W20260812 06:59:48.504664 12802 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.19.52.125:42015: Network error: Client connection negotiation failed: client connection to 127.19.52.125:42015: connect: Connection refused (error 111)
I20260812 06:59:48.572904 12802 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.19.52.125" port: 42015 } attempt: 2
I20260812 06:59:48.573920 12805 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:59:48.573610 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.125:42015 (local address 127.0.0.1:60680)
0812 06:59:48.573775 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0812 06:59:48.573841 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.52.125:42015: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":66}
I20260812 06:59:48.574719 12805 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:59:48.574483 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.125:42015 (local address 127.0.0.1:60686)
0812 06:59:48.574611 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0812 06:59:48.574659 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.52.125:42015: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260812 06:59:48.574868 12802 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.19.52.125:42015: Network error: Client connection negotiation failed: client connection to 127.19.52.125:42015: connect: Connection refused (error 111)
W20260812 06:59:48.576697 12810 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260812 06:59:48.577049 12810 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:59:48.577102 12810 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:59:48.577178 12810 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 06:59:48.581449 12810 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 06:59:48.581524 12810 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 06:59:48.581544 12810 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 06:59:48.581562 12810 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 06:59:48.581578 12810 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 06:59:48.587736 12810 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.52.84:45431
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786517687384214-19665-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786517687384214-19665-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.19.52.126:42867,127.19.52.125:42015,127.19.52.124:33027
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786517687384214-19665-0/krb5kdc/kudu_127.19.52.125.keytab
--principal=kudu/127.19.52.125
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.52.125:42015
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786517687384214-19665-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.19.52.125
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12810
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786517687384214-19665-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 06:15:54 UTC on ccycloud.quasar-rnfzce.root.comops.site
build id 4581989
I20260812 06:59:48.589079 12810 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:59:48.590221 12810 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 06:59:48.599562 12815 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:59:48.599658 12810 server_base.cc:1034] running on GCE node
W20260812 06:59:48.599562 12816 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:59:48.599599 12818 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:59:48.600440 12810 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:59:48.601687 12810 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:59:48.602913 12810 hybrid_clock.cc:648] HybridClock initialized: now 1786517988602878 us; error 60 us; skew 500 ppm
Aug 12 06:59:48 dist-test-slave-65mx krb5kdc[12709](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517988, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.52.125@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 06:59:48.606307 12810 init.cc:377] Logged in from keytab as kudu/127.19.52.125@KRBTEST.COM (short username kudu)
I20260812 06:59:48.607465 12810 webserver.cc:460] Webserver started at http://127.19.52.125:36459/ using document root <none> and password file <none>
I20260812 06:59:48.608078 12810 fs_manager.cc:359] Metadata directory not provided
I20260812 06:59:48.608175 12810 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:59:48.608433 12810 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 06:59:48.610682 12810 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786517687384214-19665-0/minicluster-data/master-1/data/instance:
uuid: "8859fc8b085a42ccbbc935c4dfaa4211"
format_stamp: "Formatted at 2026-08-12 06:59:48 on dist-test-slave-65mx"
server_key: "c692b1ea8b91504247c4aa521e5dcab8"
server_key_iv: "7f474c13952c4e138ea5d67ac285c156"
server_key_version: "encryptionkey@0"
I20260812 06:59:48.611275 12810 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786517687384214-19665-0/minicluster-data/master-1/wal/instance:
uuid: "8859fc8b085a42ccbbc935c4dfaa4211"
format_stamp: "Formatted at 2026-08-12 06:59:48 on dist-test-slave-65mx"
server_key: "c692b1ea8b91504247c4aa521e5dcab8"
server_key_iv: "7f474c13952c4e138ea5d67ac285c156"
server_key_version: "encryptionkey@0"
I20260812 06:59:48.615234 12810 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260812 06:59:48.618001 12825 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 06:59:48.619477 12810 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260812 06:59:48.619661 12810 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786517687384214-19665-0/minicluster-data/master-1/data,/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786517687384214-19665-0/minicluster-data/master-1/wal
uuid: "8859fc8b085a42ccbbc935c4dfaa4211"
format_stamp: "Formatted at 2026-08-12 06:59:48 on dist-test-slave-65mx"
server_key: "c692b1ea8b91504247c4aa521e5dcab8"
server_key_iv: "7f474c13952c4e138ea5d67ac285c156"
server_key_version: "encryptionkey@0"
I20260812 06:59:48.619804 12810 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786517687384214-19665-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786517687384214-19665-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786517687384214-19665-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:59:48.639560 12810 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 06:59:48.643353 12810 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 06:59:48.643661 12810 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:59:48.652729 12810 rpc_server.cc:307] RPC server started. Bound to: 127.19.52.125:42015
I20260812 06:59:48.653043 12885 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.52.125:42015 every 8 connection(s)
I20260812 06:59:48.653950 12810 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786517687384214-19665-0/minicluster-data/master-1/data/info.pb
I20260812 06:59:48.654759 19665 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu as pid 12810
I20260812 06:59:48.654903 19665 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786517687384214-19665-0/minicluster-data/master-1/wal/instance
I20260812 06:59:48.655236 19665 external_mini_cluster.cc:1428] Setting key ecb89bc0a1bb7a686dee80783477e092
I20260812 06:59:48.657835 12886 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 06:59:48.662199 12886 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.19.52.126" port: 42867 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.19.52.124@KRBTEST.COM; defaulting to no policy
I20260812 06:59:48.666110 12802 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.19.52.125" port: 42015 } attempt: 3
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.52.124@KRBTEST.COM" created.
Aug 12 06:59:48 dist-test-slave-65mx krb5kdc[12709](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517988, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.52.125@KRBTEST.COM for kudu/127.19.52.126@KRBTEST.COM
Aug 12 06:59:48 dist-test-slave-65mx krb5kdc[12709](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517988, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.52.126@KRBTEST.COM for kudu/127.19.52.125@KRBTEST.COM
I20260812 06:59:48.679800 12805 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:59:48.666613 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.125:42015 (local address 127.0.0.1:60702)
0812 06:59:48.666776 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0812 06:59:48.666792 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0812 06:59:48.666895 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:59:48.668622 (+ 1727us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:59:48.668625 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:59:48.668824 (+ 199us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:59:48.668957 (+ 133us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:59:48.668965 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:59:48.670632 (+ 1667us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:59:48.670638 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:59:48.671575 (+ 937us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:59:48.671586 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:59:48.672501 (+ 915us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:59:48.672506 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:59:48.672663 (+ 157us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:59:48.673122 (+ 459us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 06:59:48.673144 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:59:48.675001 (+ 1857us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 06:59:48.677903 (+ 2902us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:59:48.677909 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:59:48.677911 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:59:48.678194 (+ 283us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:59:48.678535 (+ 341us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:59:48.678538 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:59:48.678540 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:59:48.678635 (+ 95us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:59:48.679188 (+ 553us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 06:59:48.679194 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 06:59:48.679308 (+ 114us) client_negotiation.cc:769] Sending connection context
0812 06:59:48.679417 (+ 109us) client_negotiation.cc:240] Negotiation successful
0812 06:59:48.679582 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":74,"mutex_wait_us":42}
I20260812 06:59:48.680327 12807 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:59:48.664449 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35362 (local address 127.19.52.126:42867)
0812 06:59:48.664645 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0812 06:59:48.664649 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 06:59:48.665759 (+ 1110us) server_negotiation.cc:407] Connection header received
0812 06:59:48.667232 (+ 1473us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:59:48.667238 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:59:48.667305 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:59:48.667391 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:59:48.669299 (+ 1908us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:59:48.670018 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:59:48.671244 (+ 1226us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:59:48.671637 (+ 393us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:59:48.671734 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:59:48.674821 (+ 3087us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:59:48.674825 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:59:48.674830 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:59:48.674858 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:59:48.676952 (+ 2094us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:59:48.677676 (+ 724us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:59:48.677680 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:59:48.677684 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:59:48.677778 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:59:48.678161 (+ 383us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:59:48.678165 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:59:48.678167 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:59:48.678454 (+ 287us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:59:48.678548 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0812 06:59:48.680058 (+ 1510us) server_negotiation.cc:299] Negotiation successful
0812 06:59:48.680183 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":33,"server-negotiator.queue_time_us":57,"spinlock_wait_cycles":1408}
I20260812 06:59:48.681171 12892 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:59:48.666816 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60702 (local address 127.19.52.125:42015)
0812 06:59:48.668029 (+ 1213us) server_negotiation.cc:206] Beginning negotiation
0812 06:59:48.668044 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0812 06:59:48.668072 (+ 28us) server_negotiation.cc:407] Connection header received
0812 06:59:48.668121 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:59:48.668126 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:59:48.668350 (+ 224us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:59:48.668535 (+ 185us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:59:48.669123 (+ 588us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:59:48.670509 (+ 1386us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:59:48.671724 (+ 1215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:59:48.672354 (+ 630us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:59:48.672477 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:59:48.675155 (+ 2678us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:59:48.675164 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:59:48.675185 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:59:48.675207 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:59:48.677773 (+ 2566us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:59:48.678335 (+ 562us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:59:48.678340 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:59:48.678342 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:59:48.678424 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:59:48.678742 (+ 318us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:59:48.678745 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:59:48.678747 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:59:48.679048 (+ 301us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:59:48.679200 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0812 06:59:48.679594 (+ 394us) server_negotiation.cc:299] Negotiation successful
0812 06:59:48.680021 (+ 427us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":26,"server-negotiator.queue_time_us":1082,"spinlock_wait_cycles":4096,"thread_start_us":172,"threads_started":1}
I20260812 06:59:48.681171 12891 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:59:48.664968 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.126:42867 (local address 127.0.0.1:35362)
0812 06:59:48.665513 (+ 545us) negotiation.cc:107] Waiting for socket to connect
0812 06:59:48.665614 (+ 101us) client_negotiation.cc:174] Beginning negotiation
0812 06:59:48.666936 (+ 1322us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:59:48.667889 (+ 953us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:59:48.667898 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:59:48.668306 (+ 408us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:59:48.669160 (+ 854us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:59:48.669174 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:59:48.670171 (+ 997us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:59:48.670175 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:59:48.671115 (+ 940us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:59:48.671124 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:59:48.671803 (+ 679us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:59:48.671807 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:59:48.672002 (+ 195us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:59:48.672960 (+ 958us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 06:59:48.672987 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:59:48.674668 (+ 1681us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 06:59:48.677103 (+ 2435us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:59:48.677160 (+ 57us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:59:48.677173 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:59:48.677543 (+ 370us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:59:48.677887 (+ 344us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:59:48.677890 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:59:48.677892 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:59:48.678042 (+ 150us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:59:48.678606 (+ 564us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 06:59:48.678612 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 06:59:48.678958 (+ 346us) client_negotiation.cc:769] Sending connection context
0812 06:59:48.680474 (+ 1516us) client_negotiation.cc:240] Negotiation successful
0812 06:59:48.680715 (+ 241us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":335,"mutex_wait_us":7,"spinlock_wait_cycles":1280,"thread_start_us":148,"threads_started":1}
I20260812 06:59:48.682736 12886 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.19.52.125" port: 42015 } has no permanent_uuid. Determining permanent_uuid...
I20260812 06:59:48.683779 12802 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.19.52.124" port: 33027 } has no permanent_uuid. Determining permanent_uuid...
I20260812 06:59:48.684667 12805 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:59:48.684268 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.124:33027 (local address 127.0.0.1:33300)
0812 06:59:48.684413 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0812 06:59:48.684481 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.52.124:33027: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
I20260812 06:59:48.685379 12805 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:59:48.685103 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.124:33027 (local address 127.0.0.1:33302)
0812 06:59:48.685250 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0812 06:59:48.685305 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.52.124:33027: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":80}
W20260812 06:59:48.685491 12802 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.19.52.124:33027: Network error: Client connection negotiation failed: client connection to 127.19.52.124:33027: connect: Connection refused (error 111)
Aug 12 06:59:48 dist-test-slave-65mx krb5kdc[12709](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517988, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.52.125@KRBTEST.COM for kudu/127.19.52.125@KRBTEST.COM
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.52.124 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786517687384214-19665-0/krb5kdc/kudu_127.19.52.124.keytab.
Entry for principal kudu/127.19.52.124 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786517687384214-19665-0/krb5kdc/kudu_127.19.52.124.keytab.
I20260812 06:59:48.693260 19665 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu
/tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786517687384214-19665-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786517687384214-19665-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786517687384214-19665-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786517687384214-19665-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.52.124:33027
--webserver_interface=127.19.52.124
--webserver_port=0
--keytab_file=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786517687384214-19665-0/krb5kdc/kudu_127.19.52.124.keytab
--principal=kudu/127.19.52.124
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.52.84:45431
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.19.52.126:42867,127.19.52.125:42015,127.19.52.124:33027
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786517687384214-19665-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786517687384214-19665-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786517687384214-19665-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 06:59:48.693903 12892 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:59:48.683401 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60718 (local address 127.19.52.125:42015)
0812 06:59:48.683521 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0812 06:59:48.683524 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 06:59:48.683669 (+ 145us) server_negotiation.cc:407] Connection header received
0812 06:59:48.683821 (+ 152us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:59:48.683824 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:59:48.683873 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:59:48.683959 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:59:48.684675 (+ 716us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:59:48.685342 (+ 667us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:59:48.686353 (+ 1011us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:59:48.686633 (+ 280us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:59:48.686781 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:59:48.689241 (+ 2460us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:59:48.689244 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:59:48.689246 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:59:48.689271 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:59:48.691211 (+ 1940us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:59:48.691912 (+ 701us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:59:48.691918 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:59:48.691923 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:59:48.692014 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:59:48.692738 (+ 724us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:59:48.692741 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:59:48.692742 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:59:48.692945 (+ 203us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:59:48.693043 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0812 06:59:48.693481 (+ 438us) server_negotiation.cc:299] Negotiation successful
0812 06:59:48.693722 (+ 241us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":60,"server-negotiator.queue_time_us":48}
I20260812 06:59:48.693929 12891 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:59:48.683335 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.125:42015 (local address 127.0.0.1:60718)
0812 06:59:48.683584 (+ 249us) negotiation.cc:107] Waiting for socket to connect
0812 06:59:48.683605 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0812 06:59:48.683719 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:59:48.684067 (+ 348us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:59:48.684072 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:59:48.684250 (+ 178us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:59:48.684380 (+ 130us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:59:48.684387 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:59:48.685499 (+ 1112us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:59:48.685505 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:59:48.686231 (+ 726us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:59:48.686240 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:59:48.686768 (+ 528us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:59:48.686771 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:59:48.686868 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:59:48.687497 (+ 629us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 06:59:48.687518 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:59:48.689067 (+ 1549us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 06:59:48.691335 (+ 2268us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:59:48.691341 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:59:48.691344 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:59:48.691782 (+ 438us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:59:48.692152 (+ 370us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:59:48.692154 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:59:48.692156 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:59:48.692387 (+ 231us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:59:48.693061 (+ 674us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 06:59:48.693068 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 06:59:48.693193 (+ 125us) client_negotiation.cc:769] Sending connection context
0812 06:59:48.693327 (+ 134us) client_negotiation.cc:240] Negotiation successful
0812 06:59:48.693610 (+ 283us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":52}
I20260812 06:59:48.696163 12886 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.19.52.124" port: 33027 } has no permanent_uuid. Determining permanent_uuid...
I20260812 06:59:48.697168 12891 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:59:48.696844 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.124:33027 (local address 127.0.0.1:33304)
0812 06:59:48.696990 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0812 06:59:48.697097 (+ 107us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.52.124:33027: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
W20260812 06:59:48.698014 12826 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.19.52.124:33027: connect: Connection refused (error 111)
I20260812 06:59:48.699216 12891 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:59:48.698848 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.124:33027 (local address 127.0.0.1:33318)
0812 06:59:48.699071 (+ 223us) negotiation.cc:107] Waiting for socket to connect
0812 06:59:48.699142 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.52.124:33027: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":95}
W20260812 06:59:48.699661 12886 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.19.52.124:33027: Network error: Client connection negotiation failed: client connection to 127.19.52.124:33027: connect: Connection refused (error 111)
I20260812 06:59:48.716612 12802 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.19.52.124" port: 33027 } attempt: 1
I20260812 06:59:48.717639 12805 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:59:48.717282 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.124:33027 (local address 127.0.0.1:33326)
0812 06:59:48.717470 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0812 06:59:48.717544 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.52.124:33027: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":68}
I20260812 06:59:48.718479 12805 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:59:48.718230 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.124:33027 (local address 127.0.0.1:33340)
0812 06:59:48.718352 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0812 06:59:48.718410 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.52.124:33027: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260812 06:59:48.718704 12802 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.19.52.124:33027: Network error: Client connection negotiation failed: client connection to 127.19.52.124:33027: connect: Connection refused (error 111)
I20260812 06:59:48.749236 12886 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.19.52.124" port: 33027 } attempt: 1
I20260812 06:59:48.750211 12891 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:59:48.749875 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.124:33027 (local address 127.0.0.1:33346)
0812 06:59:48.750040 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0812 06:59:48.750126 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.52.124:33027: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":71}
I20260812 06:59:48.751266 12891 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:59:48.751002 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.124:33027 (local address 127.0.0.1:33354)
0812 06:59:48.751155 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0812 06:59:48.751207 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.52.124:33027: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":85}
W20260812 06:59:48.751433 12886 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.19.52.124:33027: Network error: Client connection negotiation failed: client connection to 127.19.52.124:33027: connect: Connection refused (error 111)
I20260812 06:59:48.793943 12802 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.19.52.124" port: 33027 } attempt: 2
I20260812 06:59:48.794965 12805 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:59:48.794544 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.124:33027 (local address 127.0.0.1:33366)
0812 06:59:48.794742 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0812 06:59:48.794831 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.52.124:33027: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":69}
I20260812 06:59:48.795861 12805 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:59:48.795599 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.124:33027 (local address 127.0.0.1:33380)
0812 06:59:48.795722 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0812 06:59:48.795787 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.52.124:33027: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
W20260812 06:59:48.796068 12802 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.19.52.124:33027: Network error: Client connection negotiation failed: client connection to 127.19.52.124:33027: connect: Connection refused (error 111)
W20260812 06:59:48.818586 12894 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260812 06:59:48.818912 12894 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:59:48.819007 12894 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:59:48.819051 12894 flags.cc:425] Enabled unsafe flag: --never_fsync=true
I20260812 06:59:48.819619 12886 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.19.52.124" port: 33027 } attempt: 2
I20260812 06:59:48.820588 12891 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:59:48.820229 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.124:33027 (local address 127.0.0.1:33384)
0812 06:59:48.820394 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0812 06:59:48.820460 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.52.124:33027: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":66}
I20260812 06:59:48.821556 12891 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:59:48.821228 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.124:33027 (local address 127.0.0.1:33392)
0812 06:59:48.821426 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0812 06:59:48.821495 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.52.124:33027: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":86}
W20260812 06:59:48.821703 12886 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.19.52.124:33027: Network error: Client connection negotiation failed: client connection to 127.19.52.124:33027: connect: Connection refused (error 111)
W20260812 06:59:48.824083 12894 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 06:59:48.824144 12894 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 06:59:48.824165 12894 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 06:59:48.824222 12894 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 06:59:48.824277 12894 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 06:59:48.830003 12894 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.52.84:45431
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786517687384214-19665-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786517687384214-19665-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.19.52.126:42867,127.19.52.125:42015,127.19.52.124:33027
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786517687384214-19665-0/krb5kdc/kudu_127.19.52.124.keytab
--principal=kudu/127.19.52.124
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.52.124:33027
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786517687384214-19665-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.19.52.124
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12894
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786517687384214-19665-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 06:15:54 UTC on ccycloud.quasar-rnfzce.root.comops.site
build id 4581989
I20260812 06:59:48.831205 12894 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:59:48.832291 12894 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 06:59:48.841519 12903 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:59:48.841573 12902 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:59:48.841612 12906 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:59:48.841687 12894 server_base.cc:1034] running on GCE node
I20260812 06:59:48.842442 12894 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:59:48.843628 12894 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:59:48.844813 12894 hybrid_clock.cc:648] HybridClock initialized: now 1786517988844774 us; error 54 us; skew 500 ppm
Aug 12 06:59:48 dist-test-slave-65mx krb5kdc[12709](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517988, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.52.124@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 06:59:48.848084 12894 init.cc:377] Logged in from keytab as kudu/127.19.52.124@KRBTEST.COM (short username kudu)
I20260812 06:59:48.849146 12894 webserver.cc:460] Webserver started at http://127.19.52.124:33799/ using document root <none> and password file <none>
I20260812 06:59:48.849735 12894 fs_manager.cc:359] Metadata directory not provided
I20260812 06:59:48.849833 12894 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:59:48.850077 12894 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 06:59:48.852226 12894 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786517687384214-19665-0/minicluster-data/master-2/data/instance:
uuid: "98d1bb7030104c4da7fcd66a495fdf43"
format_stamp: "Formatted at 2026-08-12 06:59:48 on dist-test-slave-65mx"
server_key: "41421b145e3de6b8ab47141b8207277f"
server_key_iv: "ed61e13cf0615cf5dcb648a02722bf38"
server_key_version: "encryptionkey@0"
I20260812 06:59:48.852843 12894 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786517687384214-19665-0/minicluster-data/master-2/wal/instance:
uuid: "98d1bb7030104c4da7fcd66a495fdf43"
format_stamp: "Formatted at 2026-08-12 06:59:48 on dist-test-slave-65mx"
server_key: "41421b145e3de6b8ab47141b8207277f"
server_key_iv: "ed61e13cf0615cf5dcb648a02722bf38"
server_key_version: "encryptionkey@0"
I20260812 06:59:48.856885 12894 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260812 06:59:48.859488 12914 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 06:59:48.861068 12894 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 06:59:48.861225 12894 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786517687384214-19665-0/minicluster-data/master-2/data,/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786517687384214-19665-0/minicluster-data/master-2/wal
uuid: "98d1bb7030104c4da7fcd66a495fdf43"
format_stamp: "Formatted at 2026-08-12 06:59:48 on dist-test-slave-65mx"
server_key: "41421b145e3de6b8ab47141b8207277f"
server_key_iv: "ed61e13cf0615cf5dcb648a02722bf38"
server_key_version: "encryptionkey@0"
I20260812 06:59:48.861352 12894 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786517687384214-19665-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786517687384214-19665-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786517687384214-19665-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:59:48.895257 12802 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.19.52.124" port: 33027 } attempt: 3
I20260812 06:59:48.896202 12805 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:59:48.895845 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.124:33027 (local address 127.0.0.1:33396)
0812 06:59:48.896016 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0812 06:59:48.896112 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.52.124:33027: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":71}
I20260812 06:59:48.897038 12805 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:59:48.896793 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.124:33027 (local address 127.0.0.1:33398)
0812 06:59:48.896910 (+ 117us) negotiation.cc:107] Waiting for socket to connect
0812 06:59:48.896975 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.52.124:33027: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260812 06:59:48.897167 12802 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.19.52.124:33027: Network error: Client connection negotiation failed: client connection to 127.19.52.124:33027: connect: Connection refused (error 111)
I20260812 06:59:48.907025 12894 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 06:59:48.910259 12894 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 06:59:48.910480 12894 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:59:48.912879 12886 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.19.52.124" port: 33027 } attempt: 3
I20260812 06:59:48.913758 12891 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:59:48.913382 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.124:33027 (local address 127.0.0.1:33408)
0812 06:59:48.913564 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0812 06:59:48.913640 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.52.124:33027: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":63}
I20260812 06:59:48.914634 12891 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:59:48.914304 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.124:33027 (local address 127.0.0.1:33412)
0812 06:59:48.914480 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0812 06:59:48.914545 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.52.124:33027: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":90}
W20260812 06:59:48.914842 12886 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.19.52.124:33027: Network error: Client connection negotiation failed: client connection to 127.19.52.124:33027: connect: Connection refused (error 111)
I20260812 06:59:48.918466 12894 rpc_server.cc:307] RPC server started. Bound to: 127.19.52.124:33027
I20260812 06:59:48.918776 12973 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.52.124:33027 every 8 connection(s)
I20260812 06:59:48.919579 12894 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786517687384214-19665-0/minicluster-data/master-2/data/info.pb
I20260812 06:59:48.922487 12974 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 06:59:48.923447 19665 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu as pid 12894
I20260812 06:59:48.923573 19665 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786517687384214-19665-0/minicluster-data/master-2/wal/instance
I20260812 06:59:48.923918 19665 external_mini_cluster.cc:1428] Setting key 6b68313e7417cc92816d3e31a82d0d55
I20260812 06:59:48.926967 12974 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.19.52.126" port: 42867 } has no permanent_uuid. Determining permanent_uuid...
Aug 12 06:59:48 dist-test-slave-65mx krb5kdc[12709](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517988, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.19.52.124@KRBTEST.COM
Aug 12 06:59:48 dist-test-slave-65mx krb5kdc[12709](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517988, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.52.124@KRBTEST.COM for kudu/127.19.52.126@KRBTEST.COM
I20260812 06:59:48.940250 12807 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:59:48.929190 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35376 (local address 127.19.52.126:42867)
0812 06:59:48.929352 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0812 06:59:48.929357 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 06:59:48.929712 (+ 355us) server_negotiation.cc:407] Connection header received
0812 06:59:48.929915 (+ 203us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:59:48.929918 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:59:48.929974 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:59:48.930053 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:59:48.930912 (+ 859us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:59:48.931429 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:59:48.932636 (+ 1207us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:59:48.933035 (+ 399us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:59:48.933170 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:59:48.936269 (+ 3099us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:59:48.936272 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:59:48.936274 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:59:48.936303 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:59:48.938266 (+ 1963us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:59:48.938830 (+ 564us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:59:48.938833 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:59:48.938834 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:59:48.938916 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:59:48.939212 (+ 296us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:59:48.939214 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:59:48.939215 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:59:48.939468 (+ 253us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:59:48.939576 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0812 06:59:48.939960 (+ 384us) server_negotiation.cc:299] Negotiation successful
0812 06:59:48.940092 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260812 06:59:48.941350 12977 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:59:48.925326 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33428 (local address 127.19.52.124:33027)
0812 06:59:48.925803 (+ 477us) server_negotiation.cc:206] Beginning negotiation
0812 06:59:48.925821 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0812 06:59:48.925867 (+ 46us) server_negotiation.cc:407] Connection header received
0812 06:59:48.926545 (+ 678us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:59:48.926562 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:59:48.926906 (+ 344us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:59:48.927187 (+ 281us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:59:48.928281 (+ 1094us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:59:48.929527 (+ 1246us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:59:48.930786 (+ 1259us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:59:48.931328 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:59:48.931459 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:59:48.934808 (+ 3349us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:59:48.934814 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:59:48.934825 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:59:48.934849 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:59:48.937514 (+ 2665us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:59:48.938127 (+ 613us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:59:48.938133 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:59:48.938138 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:59:48.938257 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:59:48.938593 (+ 336us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:59:48.938597 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:59:48.938599 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:59:48.939209 (+ 610us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:59:48.939386 (+ 177us) server_negotiation.cc:1035] Waiting for connection context
0812 06:59:48.939728 (+ 342us) server_negotiation.cc:299] Negotiation successful
0812 06:59:48.940101 (+ 373us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":312,"thread_start_us":155,"threads_started":1}
I20260812 06:59:48.941350 12978 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:59:48.929128 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.126:42867 (local address 127.0.0.1:35376)
0812 06:59:48.929566 (+ 438us) negotiation.cc:107] Waiting for socket to connect
0812 06:59:48.929619 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0812 06:59:48.929743 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:59:48.930119 (+ 376us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:59:48.930124 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:59:48.930580 (+ 456us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:59:48.930760 (+ 180us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:59:48.930771 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:59:48.931665 (+ 894us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:59:48.931668 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:59:48.932472 (+ 804us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:59:48.932482 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:59:48.933213 (+ 731us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:59:48.933216 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:59:48.933297 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:59:48.933852 (+ 555us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 06:59:48.933874 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:59:48.936081 (+ 2207us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 06:59:48.938403 (+ 2322us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:59:48.938410 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:59:48.938415 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:59:48.938724 (+ 309us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:59:48.939022 (+ 298us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:59:48.939025 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:59:48.939027 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:59:48.939118 (+ 91us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:59:48.939594 (+ 476us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 06:59:48.939600 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 06:59:48.939720 (+ 120us) client_negotiation.cc:769] Sending connection context
0812 06:59:48.940416 (+ 696us) client_negotiation.cc:240] Negotiation successful
0812 06:59:48.940668 (+ 252us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":265,"thread_start_us":157,"threads_started":1}
I20260812 06:59:48.942698 12974 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.19.52.125" port: 42015 } has no permanent_uuid. Determining permanent_uuid...
Aug 12 06:59:48 dist-test-slave-65mx krb5kdc[12709](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517988, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.52.124@KRBTEST.COM for kudu/127.19.52.125@KRBTEST.COM
Aug 12 06:59:48 dist-test-slave-65mx krb5kdc[12709](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517988, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.19.52.126@KRBTEST.COM
I20260812 06:59:48.952152 12892 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:59:48.943455 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60726 (local address 127.19.52.125:42015)
0812 06:59:48.943586 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0812 06:59:48.943591 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 06:59:48.943680 (+ 89us) server_negotiation.cc:407] Connection header received
0812 06:59:48.943830 (+ 150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:59:48.943833 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:59:48.943879 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:59:48.943973 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:59:48.944378 (+ 405us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:59:48.944957 (+ 579us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:59:48.945732 (+ 775us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:59:48.945990 (+ 258us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:59:48.946102 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:59:48.948487 (+ 2385us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:59:48.948490 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:59:48.948493 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:59:48.948529 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:59:48.950149 (+ 1620us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:59:48.950583 (+ 434us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:59:48.950587 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:59:48.950589 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:59:48.950681 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:59:48.951201 (+ 520us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:59:48.951205 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:59:48.951207 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:59:48.951477 (+ 270us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:59:48.951580 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0812 06:59:48.951866 (+ 286us) server_negotiation.cc:299] Negotiation successful
0812 06:59:48.951985 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260812 06:59:48.952152 12978 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:59:48.943275 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.125:42015 (local address 127.0.0.1:60726)
0812 06:59:48.943597 (+ 322us) negotiation.cc:107] Waiting for socket to connect
0812 06:59:48.943613 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0812 06:59:48.943723 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:59:48.943984 (+ 261us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:59:48.943987 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:59:48.944173 (+ 186us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:59:48.944256 (+ 83us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:59:48.944261 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:59:48.945068 (+ 807us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:59:48.945073 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:59:48.945610 (+ 537us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:59:48.945619 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:59:48.946121 (+ 502us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:59:48.946125 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:59:48.946217 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:59:48.946760 (+ 543us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 06:59:48.946781 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:59:48.948336 (+ 1555us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 06:59:48.950266 (+ 1930us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:59:48.950269 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:59:48.950271 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:59:48.950474 (+ 203us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:59:48.950991 (+ 517us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:59:48.951003 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:59:48.951005 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:59:48.951091 (+ 86us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:59:48.951568 (+ 477us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 06:59:48.951572 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 06:59:48.951679 (+ 107us) client_negotiation.cc:769] Sending connection context
0812 06:59:48.951808 (+ 129us) client_negotiation.cc:240] Negotiation successful
0812 06:59:48.951968 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":254}
I20260812 06:59:48.953346 12974 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.19.52.124" port: 33027 } has no permanent_uuid. Determining permanent_uuid...
I20260812 06:59:48.954133 12807 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:59:48.944066 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35386 (local address 127.19.52.126:42867)
0812 06:59:48.944184 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0812 06:59:48.944190 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 06:59:48.944205 (+ 15us) server_negotiation.cc:407] Connection header received
0812 06:59:48.944323 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:59:48.944326 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:59:48.944366 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:59:48.944461 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:59:48.945122 (+ 661us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:59:48.945613 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:59:48.946392 (+ 779us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:59:48.946703 (+ 311us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:59:48.946851 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:59:48.950401 (+ 3550us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:59:48.950405 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:59:48.950407 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:59:48.950512 (+ 105us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:59:48.952091 (+ 1579us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:59:48.952667 (+ 576us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:59:48.952671 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:59:48.952673 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:59:48.952763 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:59:48.953103 (+ 340us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:59:48.953111 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:59:48.953112 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:59:48.953298 (+ 186us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:59:48.953653 (+ 355us) server_negotiation.cc:1035] Waiting for connection context
0812 06:59:48.953820 (+ 167us) server_negotiation.cc:299] Negotiation successful
0812 06:59:48.953954 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
Aug 12 06:59:48 dist-test-slave-65mx krb5kdc[12709](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517988, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.52.124@KRBTEST.COM for kudu/127.19.52.124@KRBTEST.COM
Aug 12 06:59:48 dist-test-slave-65mx krb5kdc[12709](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517988, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.19.52.125@KRBTEST.COM
I20260812 06:59:48.962672 12978 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:59:48.954040 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.124:33027 (local address 127.0.0.1:33434)
0812 06:59:48.954144 (+ 104us) negotiation.cc:107] Waiting for socket to connect
0812 06:59:48.954160 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0812 06:59:48.954242 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:59:48.955026 (+ 784us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:59:48.955029 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:59:48.955182 (+ 153us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:59:48.955268 (+ 86us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:59:48.955273 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:59:48.956083 (+ 810us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:59:48.956089 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:59:48.956686 (+ 597us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:59:48.956695 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:59:48.957178 (+ 483us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:59:48.957181 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:59:48.957259 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:59:48.957835 (+ 576us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 06:59:48.957847 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:59:48.959286 (+ 1439us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 06:59:48.961157 (+ 1871us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:59:48.961163 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:59:48.961166 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:59:48.961361 (+ 195us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:59:48.961641 (+ 280us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:59:48.961643 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:59:48.961645 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:59:48.961729 (+ 84us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:59:48.962110 (+ 381us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 06:59:48.962113 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 06:59:48.962199 (+ 86us) client_negotiation.cc:769] Sending connection context
0812 06:59:48.962314 (+ 115us) client_negotiation.cc:240] Negotiation successful
0812 06:59:48.962466 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":31}
I20260812 06:59:48.962672 12977 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:59:48.954198 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33434 (local address 127.19.52.124:33027)
0812 06:59:48.954740 (+ 542us) server_negotiation.cc:206] Beginning negotiation
0812 06:59:48.954745 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 06:59:48.954759 (+ 14us) server_negotiation.cc:407] Connection header received
0812 06:59:48.954793 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:59:48.954797 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:59:48.954847 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:59:48.954932 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:59:48.955364 (+ 432us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:59:48.955972 (+ 608us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:59:48.956797 (+ 825us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:59:48.957063 (+ 266us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:59:48.957173 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:59:48.959430 (+ 2257us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:59:48.959434 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:59:48.959438 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:59:48.959465 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:59:48.961049 (+ 1584us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:59:48.961458 (+ 409us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:59:48.961463 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:59:48.961465 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:59:48.961542 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:59:48.961821 (+ 279us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:59:48.961824 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:59:48.961826 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:59:48.962013 (+ 187us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:59:48.962122 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0812 06:59:48.962358 (+ 236us) server_negotiation.cc:299] Negotiation successful
0812 06:59:48.962523 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":70,"server-negotiator.queue_time_us":466}
I20260812 06:59:48.964761 12892 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:59:48.955574 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60728 (local address 127.19.52.125:42015)
0812 06:59:48.955697 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0812 06:59:48.955701 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 06:59:48.955783 (+ 82us) server_negotiation.cc:407] Connection header received
0812 06:59:48.955915 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:59:48.955918 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:59:48.955958 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:59:48.956049 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:59:48.956694 (+ 645us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:59:48.957186 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:59:48.958208 (+ 1022us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:59:48.958443 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:59:48.958548 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:59:48.961267 (+ 2719us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:59:48.961271 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:59:48.961274 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:59:48.961301 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:59:48.962846 (+ 1545us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:59:48.963414 (+ 568us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:59:48.963417 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:59:48.963418 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:59:48.963495 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:59:48.963802 (+ 307us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:59:48.963805 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:59:48.963807 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:59:48.964024 (+ 217us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:59:48.964299 (+ 275us) server_negotiation.cc:1035] Waiting for connection context
0812 06:59:48.964436 (+ 137us) server_negotiation.cc:299] Negotiation successful
0812 06:59:48.964595 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260812 06:59:48.967075 12974 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 98d1bb7030104c4da7fcd66a495fdf43: Bootstrap starting.
I20260812 06:59:48.968338 19665 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 06:59:48.970028 12974 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 98d1bb7030104c4da7fcd66a495fdf43: Neither blocks nor log segments found. Creating new log.
I20260812 06:59:48.970784 12974 log.cc:824] T 00000000000000000000000000000000 P 98d1bb7030104c4da7fcd66a495fdf43: Log is configured to *not* fsync() on all Append() calls
I20260812 06:59:48.969847 12772 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "1f70bae80ad9489b91ebf7370ecdd4d9"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35386
I20260812 06:59:48.973224 12974 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 98d1bb7030104c4da7fcd66a495fdf43: No bootstrap required, opened a new log
I20260812 06:59:48.976379 12974 raft_consensus.cc:348] T 00000000000000000000000000000000 P 98d1bb7030104c4da7fcd66a495fdf43 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f70bae80ad9489b91ebf7370ecdd4d9" member_type: VOTER last_known_addr { host: "127.19.52.126" port: 42867 } } peers { permanent_uuid: "8859fc8b085a42ccbbc935c4dfaa4211" member_type: VOTER last_known_addr { host: "127.19.52.125" port: 42015 } } peers { permanent_uuid: "98d1bb7030104c4da7fcd66a495fdf43" member_type: VOTER last_known_addr { host: "127.19.52.124" port: 33027 } }
I20260812 06:59:48.976722 12974 raft_consensus.cc:729] T 00000000000000000000000000000000 P 98d1bb7030104c4da7fcd66a495fdf43 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 98d1bb7030104c4da7fcd66a495fdf43, State: Initialized, Role: FOLLOWER
I20260812 06:59:48.977069 12974 consensus_queue.cc:260] T 00000000000000000000000000000000 P 98d1bb7030104c4da7fcd66a495fdf43 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f70bae80ad9489b91ebf7370ecdd4d9" member_type: VOTER last_known_addr { host: "127.19.52.126" port: 42867 } } peers { permanent_uuid: "8859fc8b085a42ccbbc935c4dfaa4211" member_type: VOTER last_known_addr { host: "127.19.52.125" port: 42015 } } peers { permanent_uuid: "98d1bb7030104c4da7fcd66a495fdf43" member_type: VOTER last_known_addr { host: "127.19.52.124" port: 33027 } }
I20260812 06:59:48.977648 12981 sys_catalog.cc:455] T 00000000000000000000000000000000 P 98d1bb7030104c4da7fcd66a495fdf43 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f70bae80ad9489b91ebf7370ecdd4d9" member_type: VOTER last_known_addr { host: "127.19.52.126" port: 42867 } } peers { permanent_uuid: "8859fc8b085a42ccbbc935c4dfaa4211" member_type: VOTER last_known_addr { host: "127.19.52.125" port: 42015 } } peers { permanent_uuid: "98d1bb7030104c4da7fcd66a495fdf43" member_type: VOTER last_known_addr { host: "127.19.52.124" port: 33027 } } }
I20260812 06:59:48.977807 12981 sys_catalog.cc:458] T 00000000000000000000000000000000 P 98d1bb7030104c4da7fcd66a495fdf43 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 06:59:48.978714 12974 sys_catalog.cc:564] T 00000000000000000000000000000000 P 98d1bb7030104c4da7fcd66a495fdf43 [sys.catalog]: configured and running, proceeding with master startup.
W20260812 06:59:48.984515 12992 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 98d1bb7030104c4da7fcd66a495fdf43: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 06:59:48.984699 12992 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 06:59:48.983301 12894 master_runner.cc:186] Error getting master registration for 127.19.52.126:42867: OK, instance_id { permanent_uuid: "1f70bae80ad9489b91ebf7370ecdd4d9" instance_seqno: 1786517988418819 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260812 06:59:48.986977 12894 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260812 06:59:49.058050 12886 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.19.52.124" port: 33027 } attempt: 4
I20260812 06:59:49.061308 12802 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.19.52.124" port: 33027 } attempt: 4
Aug 12 06:59:49 dist-test-slave-65mx krb5kdc[12709](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517988, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.52.125@KRBTEST.COM for kudu/127.19.52.124@KRBTEST.COM
Aug 12 06:59:49 dist-test-slave-65mx krb5kdc[12709](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517988, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.52.126@KRBTEST.COM for kudu/127.19.52.124@KRBTEST.COM
I20260812 06:59:49.069299 12891 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:59:49.058699 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.124:33027 (local address 127.0.0.1:33442)
0812 06:59:49.058871 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0812 06:59:49.058888 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0812 06:59:49.058995 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:59:49.059282 (+ 287us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:59:49.059285 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:59:49.059512 (+ 227us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:59:49.059641 (+ 129us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:59:49.059648 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:59:49.060747 (+ 1099us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:59:49.060751 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:59:49.061575 (+ 824us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:59:49.061588 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:59:49.062252 (+ 664us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:59:49.062255 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:59:49.062339 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:59:49.063209 (+ 870us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 06:59:49.063226 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:59:49.064978 (+ 1752us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 06:59:49.067338 (+ 2360us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:59:49.067342 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:59:49.067344 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:59:49.067617 (+ 273us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:59:49.067999 (+ 382us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:59:49.068003 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:59:49.068005 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:59:49.068118 (+ 113us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:59:49.068680 (+ 562us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 06:59:49.068686 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 06:59:49.068802 (+ 116us) client_negotiation.cc:769] Sending connection context
0812 06:59:49.068918 (+ 116us) client_negotiation.cc:240] Negotiation successful
0812 06:59:49.069094 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":62}
I20260812 06:59:49.069536 12977 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:59:49.058782 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33442 (local address 127.19.52.124:33027)
0812 06:59:49.058948 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0812 06:59:49.058954 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 06:59:49.058973 (+ 19us) server_negotiation.cc:407] Connection header received
0812 06:59:49.059091 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:59:49.059097 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:59:49.059151 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:59:49.059244 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:59:49.059765 (+ 521us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:59:49.060616 (+ 851us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:59:49.061734 (+ 1118us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:59:49.062026 (+ 292us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:59:49.062121 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:59:49.065116 (+ 2995us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:59:49.065120 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:59:49.065122 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:59:49.065149 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:59:49.067219 (+ 2070us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:59:49.067735 (+ 516us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:59:49.067739 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:59:49.067740 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:59:49.067829 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:59:49.068230 (+ 401us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:59:49.068234 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:59:49.068235 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:59:49.068589 (+ 354us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:59:49.068713 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0812 06:59:49.069014 (+ 301us) server_negotiation.cc:299] Negotiation successful
0812 06:59:49.069255 (+ 241us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57,"spinlock_wait_cycles":39040}
I20260812 06:59:49.072700 12805 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:59:49.061823 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.124:33027 (local address 127.0.0.1:33450)
0812 06:59:49.061955 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0812 06:59:49.061979 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0812 06:59:49.062066 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:59:49.062710 (+ 644us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:59:49.062714 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:59:49.062922 (+ 208us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:59:49.063031 (+ 109us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:59:49.063039 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:59:49.064498 (+ 1459us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:59:49.064505 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:59:49.065357 (+ 852us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:59:49.065369 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:59:49.066052 (+ 683us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:59:49.066057 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:59:49.066163 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:59:49.066794 (+ 631us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 06:59:49.066812 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:59:49.068250 (+ 1438us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 06:59:49.071050 (+ 2800us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:59:49.071053 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:59:49.071055 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:59:49.071311 (+ 256us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:59:49.071628 (+ 317us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:59:49.071632 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:59:49.071634 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:59:49.071730 (+ 96us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:59:49.072144 (+ 414us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 06:59:49.072147 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 06:59:49.072265 (+ 118us) client_negotiation.cc:769] Sending connection context
0812 06:59:49.072367 (+ 102us) client_negotiation.cc:240] Negotiation successful
0812 06:59:49.072511 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":34}
I20260812 06:59:49.073120 12886 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8859fc8b085a42ccbbc935c4dfaa4211: Bootstrap starting.
I20260812 06:59:49.073253 12994 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:59:49.062021 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33450 (local address 127.19.52.124:33027)
0812 06:59:49.062447 (+ 426us) server_negotiation.cc:206] Beginning negotiation
0812 06:59:49.062462 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0812 06:59:49.062475 (+ 13us) server_negotiation.cc:407] Connection header received
0812 06:59:49.062509 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:59:49.062514 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:59:49.062568 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:59:49.062660 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:59:49.063161 (+ 501us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:59:49.064346 (+ 1185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:59:49.065555 (+ 1209us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:59:49.065925 (+ 370us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:59:49.066028 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:59:49.068415 (+ 2387us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:59:49.068420 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:59:49.068423 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:59:49.068472 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:59:49.070934 (+ 2462us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:59:49.071416 (+ 482us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:59:49.071420 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:59:49.071422 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:59:49.071513 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:59:49.071820 (+ 307us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:59:49.071824 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:59:49.071840 (+ 16us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:59:49.072050 (+ 210us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:59:49.072141 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0812 06:59:49.072485 (+ 344us) server_negotiation.cc:299] Negotiation successful
0812 06:59:49.072727 (+ 242us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":355,"thread_start_us":146,"threads_started":1}
I20260812 06:59:49.076514 12886 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8859fc8b085a42ccbbc935c4dfaa4211: Neither blocks nor log segments found. Creating new log.
I20260812 06:59:49.077376 12886 log.cc:824] T 00000000000000000000000000000000 P 8859fc8b085a42ccbbc935c4dfaa4211: Log is configured to *not* fsync() on all Append() calls
I20260812 06:59:49.079947 12886 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8859fc8b085a42ccbbc935c4dfaa4211: No bootstrap required, opened a new log
I20260812 06:59:49.081220 12802 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1f70bae80ad9489b91ebf7370ecdd4d9: Bootstrap starting.
I20260812 06:59:49.084188 12886 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8859fc8b085a42ccbbc935c4dfaa4211 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f70bae80ad9489b91ebf7370ecdd4d9" member_type: VOTER last_known_addr { host: "127.19.52.126" port: 42867 } } peers { permanent_uuid: "8859fc8b085a42ccbbc935c4dfaa4211" member_type: VOTER last_known_addr { host: "127.19.52.125" port: 42015 } } peers { permanent_uuid: "98d1bb7030104c4da7fcd66a495fdf43" member_type: VOTER last_known_addr { host: "127.19.52.124" port: 33027 } }
I20260812 06:59:49.084378 12802 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1f70bae80ad9489b91ebf7370ecdd4d9: Neither blocks nor log segments found. Creating new log.
I20260812 06:59:49.084447 12886 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8859fc8b085a42ccbbc935c4dfaa4211 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8859fc8b085a42ccbbc935c4dfaa4211, State: Initialized, Role: FOLLOWER
I20260812 06:59:49.084995 12886 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8859fc8b085a42ccbbc935c4dfaa4211 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f70bae80ad9489b91ebf7370ecdd4d9" member_type: VOTER last_known_addr { host: "127.19.52.126" port: 42867 } } peers { permanent_uuid: "8859fc8b085a42ccbbc935c4dfaa4211" member_type: VOTER last_known_addr { host: "127.19.52.125" port: 42015 } } peers { permanent_uuid: "98d1bb7030104c4da7fcd66a495fdf43" member_type: VOTER last_known_addr { host: "127.19.52.124" port: 33027 } }
I20260812 06:59:49.085422 12802 log.cc:824] T 00000000000000000000000000000000 P 1f70bae80ad9489b91ebf7370ecdd4d9: Log is configured to *not* fsync() on all Append() calls
I20260812 06:59:49.085870 12996 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8859fc8b085a42ccbbc935c4dfaa4211 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f70bae80ad9489b91ebf7370ecdd4d9" member_type: VOTER last_known_addr { host: "127.19.52.126" port: 42867 } } peers { permanent_uuid: "8859fc8b085a42ccbbc935c4dfaa4211" member_type: VOTER last_known_addr { host: "127.19.52.125" port: 42015 } } peers { permanent_uuid: "98d1bb7030104c4da7fcd66a495fdf43" member_type: VOTER last_known_addr { host: "127.19.52.124" port: 33027 } } }
I20260812 06:59:49.086243 12996 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8859fc8b085a42ccbbc935c4dfaa4211 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 06:59:49.086810 12886 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8859fc8b085a42ccbbc935c4dfaa4211 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 06:59:49.087915 12802 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1f70bae80ad9489b91ebf7370ecdd4d9: No bootstrap required, opened a new log
I20260812 06:59:49.091678 12802 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1f70bae80ad9489b91ebf7370ecdd4d9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f70bae80ad9489b91ebf7370ecdd4d9" member_type: VOTER last_known_addr { host: "127.19.52.126" port: 42867 } } peers { permanent_uuid: "8859fc8b085a42ccbbc935c4dfaa4211" member_type: VOTER last_known_addr { host: "127.19.52.125" port: 42015 } } peers { permanent_uuid: "98d1bb7030104c4da7fcd66a495fdf43" member_type: VOTER last_known_addr { host: "127.19.52.124" port: 33027 } }
I20260812 06:59:49.091944 12802 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1f70bae80ad9489b91ebf7370ecdd4d9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1f70bae80ad9489b91ebf7370ecdd4d9, State: Initialized, Role: FOLLOWER
I20260812 06:59:49.092345 12802 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1f70bae80ad9489b91ebf7370ecdd4d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f70bae80ad9489b91ebf7370ecdd4d9" member_type: VOTER last_known_addr { host: "127.19.52.126" port: 42867 } } peers { permanent_uuid: "8859fc8b085a42ccbbc935c4dfaa4211" member_type: VOTER last_known_addr { host: "127.19.52.125" port: 42015 } } peers { permanent_uuid: "98d1bb7030104c4da7fcd66a495fdf43" member_type: VOTER last_known_addr { host: "127.19.52.124" port: 33027 } }
W20260812 06:59:49.093434 13011 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 8859fc8b085a42ccbbc935c4dfaa4211: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 06:59:49.093577 13011 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 06:59:49.093888 12802 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1f70bae80ad9489b91ebf7370ecdd4d9 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 06:59:49.090987 12810 master_runner.cc:186] Error getting master registration for 127.19.52.126:42867: OK, instance_id { permanent_uuid: "1f70bae80ad9489b91ebf7370ecdd4d9" instance_seqno: 1786517988418819 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260812 06:59:49.094408 12810 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260812 06:59:49.094564 12772 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1f70bae80ad9489b91ebf7370ecdd4d9 [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260812 06:59:49.094687 12772 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1f70bae80ad9489b91ebf7370ecdd4d9 [term 0 FOLLOWER]: Advancing to term 1
I20260812 06:59:49.096292 12772 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1f70bae80ad9489b91ebf7370ecdd4d9 [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f70bae80ad9489b91ebf7370ecdd4d9" member_type: VOTER last_known_addr { host: "127.19.52.126" port: 42867 } } peers { permanent_uuid: "8859fc8b085a42ccbbc935c4dfaa4211" member_type: VOTER last_known_addr { host: "127.19.52.125" port: 42015 } } peers { permanent_uuid: "98d1bb7030104c4da7fcd66a495fdf43" member_type: VOTER last_known_addr { host: "127.19.52.124" port: 33027 } }
I20260812 06:59:49.097352 12772 leader_election.cc:288] T 00000000000000000000000000000000 P 1f70bae80ad9489b91ebf7370ecdd4d9 [CANDIDATE]: Term 1 election: Requested vote from peers 8859fc8b085a42ccbbc935c4dfaa4211 (127.19.52.125:42015), 98d1bb7030104c4da7fcd66a495fdf43 (127.19.52.124:33027)
I20260812 06:59:49.098623 12944 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "1f70bae80ad9489b91ebf7370ecdd4d9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "98d1bb7030104c4da7fcd66a495fdf43"
I20260812 06:59:49.098934 12944 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 98d1bb7030104c4da7fcd66a495fdf43 [term 0 FOLLOWER]: Advancing to term 1
I20260812 06:59:49.100526 12944 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 98d1bb7030104c4da7fcd66a495fdf43 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f70bae80ad9489b91ebf7370ecdd4d9 in term 1.
I20260812 06:59:49.101159 12857 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "1f70bae80ad9489b91ebf7370ecdd4d9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "8859fc8b085a42ccbbc935c4dfaa4211"
I20260812 06:59:49.101241 12742 leader_election.cc:302] T 00000000000000000000000000000000 P 1f70bae80ad9489b91ebf7370ecdd4d9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f70bae80ad9489b91ebf7370ecdd4d9, 98d1bb7030104c4da7fcd66a495fdf43; no voters:
I20260812 06:59:49.101442 12857 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8859fc8b085a42ccbbc935c4dfaa4211 [term 0 FOLLOWER]: Advancing to term 1
I20260812 06:59:49.102859 12857 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 8859fc8b085a42ccbbc935c4dfaa4211 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f70bae80ad9489b91ebf7370ecdd4d9 in term 1.
I20260812 06:59:49.106284 13013 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1f70bae80ad9489b91ebf7370ecdd4d9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f70bae80ad9489b91ebf7370ecdd4d9" member_type: VOTER last_known_addr { host: "127.19.52.126" port: 42867 } } peers { permanent_uuid: "8859fc8b085a42ccbbc935c4dfaa4211" member_type: VOTER last_known_addr { host: "127.19.52.125" port: 42015 } } peers { permanent_uuid: "98d1bb7030104c4da7fcd66a495fdf43" member_type: VOTER last_known_addr { host: "127.19.52.124" port: 33027 } } }
I20260812 06:59:49.106448 13013 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1f70bae80ad9489b91ebf7370ecdd4d9 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 06:59:49.106551 13013 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1f70bae80ad9489b91ebf7370ecdd4d9 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 06:59:49.106642 13013 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1f70bae80ad9489b91ebf7370ecdd4d9 [term 1 LEADER]: Becoming Leader. State: Replica: 1f70bae80ad9489b91ebf7370ecdd4d9, State: Running, Role: LEADER
I20260812 06:59:49.107016 13013 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1f70bae80ad9489b91ebf7370ecdd4d9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f70bae80ad9489b91ebf7370ecdd4d9" member_type: VOTER last_known_addr { host: "127.19.52.126" port: 42867 } } peers { permanent_uuid: "8859fc8b085a42ccbbc935c4dfaa4211" member_type: VOTER last_known_addr { host: "127.19.52.125" port: 42015 } } peers { permanent_uuid: "98d1bb7030104c4da7fcd66a495fdf43" member_type: VOTER last_known_addr { host: "127.19.52.124" port: 33027 } }
I20260812 06:59:49.109694 13013 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1f70bae80ad9489b91ebf7370ecdd4d9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1f70bae80ad9489b91ebf7370ecdd4d9. Latest consensus state: current_term: 1 leader_uuid: "1f70bae80ad9489b91ebf7370ecdd4d9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f70bae80ad9489b91ebf7370ecdd4d9" member_type: VOTER last_known_addr { host: "127.19.52.126" port: 42867 } } peers { permanent_uuid: "8859fc8b085a42ccbbc935c4dfaa4211" member_type: VOTER last_known_addr { host: "127.19.52.125" port: 42015 } } peers { permanent_uuid: "98d1bb7030104c4da7fcd66a495fdf43" member_type: VOTER last_known_addr { host: "127.19.52.124" port: 33027 } } }
I20260812 06:59:49.109848 13013 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1f70bae80ad9489b91ebf7370ecdd4d9 [sys.catalog]: This master's current role is: LEADER
I20260812 06:59:49.112241 12724 master_runner.cc:263] No leader master found from master 1f70bae80ad9489b91ebf7370ecdd4d9
I20260812 06:59:49.112344 12724 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260812 06:59:49.113778 13024 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 1f70bae80ad9489b91ebf7370ecdd4d9: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 06:59:49.113864 13024 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 06:59:49.114198 13034 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 06:59:49.115242 13034 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 06:59:49.121491 12944 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 98d1bb7030104c4da7fcd66a495fdf43 [term 1 FOLLOWER]: Refusing update from remote peer 1f70bae80ad9489b91ebf7370ecdd4d9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 06:59:49.122314 13013 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 1f70bae80ad9489b91ebf7370ecdd4d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "98d1bb7030104c4da7fcd66a495fdf43" member_type: VOTER last_known_addr { host: "127.19.52.124" port: 33027 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 06:59:49.126785 12892 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:59:49.105256 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60738 (local address 127.19.52.125:42015)
0812 06:59:49.105392 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0812 06:59:49.105397 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 06:59:49.109828 (+ 4431us) server_negotiation.cc:407] Connection header received
0812 06:59:49.116849 (+ 7021us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:59:49.116854 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:59:49.116919 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:59:49.117010 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:59:49.118132 (+ 1122us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:59:49.118874 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:59:49.120056 (+ 1182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:59:49.120370 (+ 314us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:59:49.120448 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:59:49.122458 (+ 2010us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:59:49.122462 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:59:49.122465 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:59:49.122495 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:59:49.124581 (+ 2086us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:59:49.125329 (+ 748us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:59:49.125332 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:59:49.125334 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:59:49.125426 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:59:49.125752 (+ 326us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:59:49.125757 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:59:49.125762 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:59:49.126003 (+ 241us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:59:49.126111 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0812 06:59:49.126475 (+ 364us) server_negotiation.cc:299] Negotiation successful
0812 06:59:49.126624 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260812 06:59:49.126865 12981 sys_catalog.cc:455] T 00000000000000000000000000000000 P 98d1bb7030104c4da7fcd66a495fdf43 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1f70bae80ad9489b91ebf7370ecdd4d9. Latest consensus state: current_term: 1 leader_uuid: "1f70bae80ad9489b91ebf7370ecdd4d9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f70bae80ad9489b91ebf7370ecdd4d9" member_type: VOTER last_known_addr { host: "127.19.52.126" port: 42867 } } peers { permanent_uuid: "8859fc8b085a42ccbbc935c4dfaa4211" member_type: VOTER last_known_addr { host: "127.19.52.125" port: 42015 } } peers { permanent_uuid: "98d1bb7030104c4da7fcd66a495fdf43" member_type: VOTER last_known_addr { host: "127.19.52.124" port: 33027 } } }
I20260812 06:59:49.127004 12981 sys_catalog.cc:458] T 00000000000000000000000000000000 P 98d1bb7030104c4da7fcd66a495fdf43 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 06:59:49.127828 12994 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:59:49.105353 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33466 (local address 127.19.52.124:33027)
0812 06:59:49.105480 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0812 06:59:49.105484 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 06:59:49.117939 (+ 12455us) server_negotiation.cc:407] Connection header received
0812 06:59:49.118142 (+ 203us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:59:49.118146 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:59:49.118197 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:59:49.118246 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:59:49.119235 (+ 989us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:59:49.119978 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:59:49.121177 (+ 1199us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:59:49.121527 (+ 350us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:59:49.121624 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:59:49.123742 (+ 2118us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:59:49.123746 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:59:49.123749 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:59:49.123778 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:59:49.125536 (+ 1758us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:59:49.126055 (+ 519us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:59:49.126059 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:59:49.126060 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:59:49.126159 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:59:49.126497 (+ 338us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:59:49.126500 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:59:49.126502 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:59:49.126723 (+ 221us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:59:49.126828 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0812 06:59:49.127538 (+ 710us) server_negotiation.cc:299] Negotiation successful
0812 06:59:49.127676 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260812 06:59:49.128161 12857 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 8859fc8b085a42ccbbc935c4dfaa4211 [term 1 FOLLOWER]: Refusing update from remote peer 1f70bae80ad9489b91ebf7370ecdd4d9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 06:59:49.129082 13025 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 1f70bae80ad9489b91ebf7370ecdd4d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8859fc8b085a42ccbbc935c4dfaa4211" member_type: VOTER last_known_addr { host: "127.19.52.125" port: 42015 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 06:59:49.132939 12996 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8859fc8b085a42ccbbc935c4dfaa4211 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1f70bae80ad9489b91ebf7370ecdd4d9. Latest consensus state: current_term: 1 leader_uuid: "1f70bae80ad9489b91ebf7370ecdd4d9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f70bae80ad9489b91ebf7370ecdd4d9" member_type: VOTER last_known_addr { host: "127.19.52.126" port: 42867 } } peers { permanent_uuid: "8859fc8b085a42ccbbc935c4dfaa4211" member_type: VOTER last_known_addr { host: "127.19.52.125" port: 42015 } } peers { permanent_uuid: "98d1bb7030104c4da7fcd66a495fdf43" member_type: VOTER last_known_addr { host: "127.19.52.124" port: 33027 } } }
I20260812 06:59:49.133093 12996 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8859fc8b085a42ccbbc935c4dfaa4211 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 06:59:49.138069 12807 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:59:49.117328 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35396 (local address 127.19.52.126:42867)
0812 06:59:49.117456 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0812 06:59:49.117463 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 06:59:49.128347 (+ 10884us) server_negotiation.cc:407] Connection header received
0812 06:59:49.128576 (+ 229us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:59:49.128580 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:59:49.128638 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:59:49.128694 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:59:49.129625 (+ 931us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:59:49.130315 (+ 690us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:59:49.131385 (+ 1070us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:59:49.131731 (+ 346us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:59:49.131842 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:59:49.134117 (+ 2275us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:59:49.134122 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:59:49.134125 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:59:49.134155 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:59:49.136023 (+ 1868us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:59:49.136565 (+ 542us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:59:49.136570 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:59:49.136571 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:59:49.136679 (+ 108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:59:49.137040 (+ 361us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:59:49.137044 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:59:49.137045 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:59:49.137280 (+ 235us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:59:49.137385 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0812 06:59:49.137780 (+ 395us) server_negotiation.cc:299] Negotiation successful
0812 06:59:49.137917 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260812 06:59:49.141135 13013 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1f70bae80ad9489b91ebf7370ecdd4d9 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "1f70bae80ad9489b91ebf7370ecdd4d9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f70bae80ad9489b91ebf7370ecdd4d9" member_type: VOTER last_known_addr { host: "127.19.52.126" port: 42867 } } peers { permanent_uuid: "8859fc8b085a42ccbbc935c4dfaa4211" member_type: VOTER last_known_addr { host: "127.19.52.125" port: 42015 } } peers { permanent_uuid: "98d1bb7030104c4da7fcd66a495fdf43" member_type: VOTER last_known_addr { host: "127.19.52.124" port: 33027 } } }
I20260812 06:59:49.141288 13013 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1f70bae80ad9489b91ebf7370ecdd4d9 [sys.catalog]: This master's current role is: LEADER
I20260812 06:59:49.143100 13040 mvcc.cc:205] Tried to move back new op lower bound from 7317577683432574976 to 7317577683387154432. Current Snapshot: MvccSnapshot[applied={T|T < 7317577683432574976}]
I20260812 06:59:49.144905 12996 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8859fc8b085a42ccbbc935c4dfaa4211 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "1f70bae80ad9489b91ebf7370ecdd4d9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f70bae80ad9489b91ebf7370ecdd4d9" member_type: VOTER last_known_addr { host: "127.19.52.126" port: 42867 } } peers { permanent_uuid: "8859fc8b085a42ccbbc935c4dfaa4211" member_type: VOTER last_known_addr { host: "127.19.52.125" port: 42015 } } peers { permanent_uuid: "98d1bb7030104c4da7fcd66a495fdf43" member_type: VOTER last_known_addr { host: "127.19.52.124" port: 33027 } } }
I20260812 06:59:49.145058 12996 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8859fc8b085a42ccbbc935c4dfaa4211 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 06:59:49.148241 13037 mvcc.cc:205] Tried to move back new op lower bound from 7317577683432574976 to 7317577683387154432. Current Snapshot: MvccSnapshot[applied={T|T < 7317577683432574976}]
I20260812 06:59:49.148677 13013 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1f70bae80ad9489b91ebf7370ecdd4d9 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "1f70bae80ad9489b91ebf7370ecdd4d9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f70bae80ad9489b91ebf7370ecdd4d9" member_type: VOTER last_known_addr { host: "127.19.52.126" port: 42867 } } peers { permanent_uuid: "8859fc8b085a42ccbbc935c4dfaa4211" member_type: VOTER last_known_addr { host: "127.19.52.125" port: 42015 } } peers { permanent_uuid: "98d1bb7030104c4da7fcd66a495fdf43" member_type: VOTER last_known_addr { host: "127.19.52.124" port: 33027 } } }
I20260812 06:59:49.148800 13013 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1f70bae80ad9489b91ebf7370ecdd4d9 [sys.catalog]: This master's current role is: LEADER
I20260812 06:59:49.150187 12981 sys_catalog.cc:455] T 00000000000000000000000000000000 P 98d1bb7030104c4da7fcd66a495fdf43 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "1f70bae80ad9489b91ebf7370ecdd4d9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f70bae80ad9489b91ebf7370ecdd4d9" member_type: VOTER last_known_addr { host: "127.19.52.126" port: 42867 } } peers { permanent_uuid: "8859fc8b085a42ccbbc935c4dfaa4211" member_type: VOTER last_known_addr { host: "127.19.52.125" port: 42015 } } peers { permanent_uuid: "98d1bb7030104c4da7fcd66a495fdf43" member_type: VOTER last_known_addr { host: "127.19.52.124" port: 33027 } } }
I20260812 06:59:49.150321 12981 sys_catalog.cc:458] T 00000000000000000000000000000000 P 98d1bb7030104c4da7fcd66a495fdf43 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 06:59:49.150336 13034 catalog_manager.cc:1347] Generated new cluster ID: 18cdd7e4effe488d960bb9b3ccb8c8a5
I20260812 06:59:49.150406 13034 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 06:59:49.151033 13038 mvcc.cc:205] Tried to move back new op lower bound from 7317577683432574976 to 7317577683387154432. Current Snapshot: MvccSnapshot[applied={T|T < 7317577683432574976}]
I20260812 06:59:49.178025 13034 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 06:59:49.179334 13034 catalog_manager.cc:1504] Loading token signing keys...
I20260812 06:59:49.190388 13034 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1f70bae80ad9489b91ebf7370ecdd4d9: Generated new TSK 0
I20260812 06:59:49.191483 13034 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260812 06:59:49.217126 12807 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 06:59:49.214674 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35406 (local address 127.19.52.126:42867)
0812 06:59:49.214868 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0812 06:59:49.214874 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 06:59:49.216128 (+ 1254us) server_negotiation.cc:407] Connection header received
0812 06:59:49.216252 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:59:49.216256 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:59:49.216313 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:59:49.216413 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:59:49.216990 (+ 577us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:35406: BlockingRecv error: recv got EOF from 127.0.0.1:35406 (error 108)
Metrics: {"server-negotiator.queue_time_us":52}
W20260812 06:59:49.217170 12994 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 06:59:49.215037 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33468 (local address 127.19.52.124:33027)
0812 06:59:49.215178 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0812 06:59:49.215183 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 06:59:49.215764 (+ 581us) server_negotiation.cc:407] Connection header received
0812 06:59:49.215941 (+ 177us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:59:49.215945 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:59:49.216005 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:59:49.216090 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:59:49.216961 (+ 871us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:33468: BlockingRecv error: recv got EOF from 127.0.0.1:33468 (error 108)
Metrics: {"server-negotiator.queue_time_us":59}
W20260812 06:59:49.217811 12892 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 06:59:49.215313 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60750 (local address 127.19.52.125:42015)
0812 06:59:49.215486 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0812 06:59:49.215491 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 06:59:49.216525 (+ 1034us) server_negotiation.cc:407] Connection header received
0812 06:59:49.216811 (+ 286us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:59:49.216815 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:59:49.216865 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:59:49.216922 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:59:49.217678 (+ 756us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:60750: BlockingRecv error: recv got EOF from 127.0.0.1:60750 (error 108)
Metrics: {"server-negotiator.queue_time_us":70}
W20260812 06:59:49.221583 12892 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 06:59:49.220114 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60764 (local address 127.19.52.125:42015)
0812 06:59:49.220247 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0812 06:59:49.220251 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 06:59:49.220703 (+ 452us) server_negotiation.cc:407] Connection header received
0812 06:59:49.220895 (+ 192us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:59:49.220898 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:59:49.220973 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:59:49.221062 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:59:49.221479 (+ 417us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:60764: BlockingRecv error: recv got EOF from 127.0.0.1:60764 (error 108)
Metrics: {"server-negotiator.queue_time_us":48}
W20260812 06:59:49.226097 12892 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 06:59:49.225236 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60776 (local address 127.19.52.125:42015)
0812 06:59:49.225353 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0812 06:59:49.225357 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 06:59:49.225369 (+ 12us) server_negotiation.cc:407] Connection header received
0812 06:59:49.225422 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:59:49.225424 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:59:49.225464 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:59:49.225557 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:59:49.225985 (+ 428us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:60776: BlockingRecv error: recv got EOF from 127.0.0.1:60776 (error 108)
Metrics: {"server-negotiator.queue_time_us":46}
W20260812 06:59:49.231266 12892 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 06:59:49.230033 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60784 (local address 127.19.52.125:42015)
0812 06:59:49.230152 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0812 06:59:49.230156 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 06:59:49.230417 (+ 261us) server_negotiation.cc:407] Connection header received
0812 06:59:49.230572 (+ 155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:59:49.230576 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:59:49.230626 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:59:49.230716 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:59:49.231147 (+ 431us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:60784: BlockingRecv error: recv got EOF from 127.0.0.1:60784 (error 108)
Metrics: {"server-negotiator.queue_time_us":49}
W20260812 06:59:49.239144 12892 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 06:59:49.237732 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60788 (local address 127.19.52.125:42015)
0812 06:59:49.237882 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0812 06:59:49.237887 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 06:59:49.238181 (+ 294us) server_negotiation.cc:407] Connection header received
0812 06:59:49.238399 (+ 218us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:59:49.238402 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:59:49.238460 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:59:49.238548 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:59:49.239003 (+ 455us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:60788: BlockingRecv error: recv got EOF from 127.0.0.1:60788 (error 108)
Metrics: {"server-negotiator.queue_time_us":57}
W20260812 06:59:49.250696 12892 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 06:59:49.249407 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60796 (local address 127.19.52.125:42015)
0812 06:59:49.249561 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0812 06:59:49.249566 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 06:59:49.249777 (+ 211us) server_negotiation.cc:407] Connection header received
0812 06:59:49.249948 (+ 171us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:59:49.249951 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:59:49.250012 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:59:49.250102 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:59:49.250575 (+ 473us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:60796: BlockingRecv error: recv got EOF from 127.0.0.1:60796 (error 108)
Metrics: {"server-negotiator.queue_time_us":69}
W20260812 06:59:49.270577 12892 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 06:59:49.269060 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60798 (local address 127.19.52.125:42015)
0812 06:59:49.269213 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0812 06:59:49.269218 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 06:59:49.269545 (+ 327us) server_negotiation.cc:407] Connection header received
0812 06:59:49.269726 (+ 181us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:59:49.269730 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:59:49.269788 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:59:49.269886 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:59:49.270400 (+ 514us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:60798: BlockingRecv error: recv got EOF from 127.0.0.1:60798 (error 108)
Metrics: {"server-negotiator.queue_time_us":61}
W20260812 06:59:49.306280 12892 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 06:59:49.304912 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60800 (local address 127.19.52.125:42015)
0812 06:59:49.305053 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0812 06:59:49.305058 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 06:59:49.305336 (+ 278us) server_negotiation.cc:407] Connection header received
0812 06:59:49.305511 (+ 175us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:59:49.305514 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:59:49.305572 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:59:49.305668 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:59:49.306147 (+ 479us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:60800: BlockingRecv error: recv got EOF from 127.0.0.1:60800 (error 108)
Metrics: {"server-negotiator.queue_time_us":58}
W20260812 06:59:49.374222 12892 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 06:59:49.372814 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60804 (local address 127.19.52.125:42015)
0812 06:59:49.372989 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0812 06:59:49.372994 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 06:59:49.373142 (+ 148us) server_negotiation.cc:407] Connection header received
0812 06:59:49.373339 (+ 197us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:59:49.373343 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:59:49.373403 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:59:49.373514 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:59:49.374077 (+ 563us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:60804: BlockingRecv error: recv got EOF from 127.0.0.1:60804 (error 108)
Metrics: {"server-negotiator.queue_time_us":65}
W20260812 06:59:49.506031 12892 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 06:59:49.504612 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60810 (local address 127.19.52.125:42015)
0812 06:59:49.504796 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0812 06:59:49.504802 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 06:59:49.504934 (+ 132us) server_negotiation.cc:407] Connection header received
0812 06:59:49.505144 (+ 210us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:59:49.505147 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:59:49.505209 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:59:49.505311 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:59:49.505868 (+ 557us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:60810: BlockingRecv error: recv got EOF from 127.0.0.1:60810 (error 108)
Metrics: {"server-negotiator.queue_time_us":71}
W20260812 06:59:49.765827 12892 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 06:59:49.764402 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60816 (local address 127.19.52.125:42015)
0812 06:59:49.764592 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0812 06:59:49.764599 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 06:59:49.764739 (+ 140us) server_negotiation.cc:407] Connection header received
0812 06:59:49.764827 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:59:49.764829 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:59:49.764884 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:59:49.764965 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:59:49.765682 (+ 717us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:60816: BlockingRecv error: recv got EOF from 127.0.0.1:60816 (error 108)
Metrics: {"server-negotiator.queue_time_us":49}
I20260812 06:59:49.986380 12992 catalog_manager.cc:1259] Loaded cluster ID: 18cdd7e4effe488d960bb9b3ccb8c8a5
I20260812 06:59:49.986518 12992 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 98d1bb7030104c4da7fcd66a495fdf43: loading cluster ID for follower catalog manager: success
I20260812 06:59:49.988884 12992 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 98d1bb7030104c4da7fcd66a495fdf43: acquiring CA information for follower catalog manager: success
I20260812 06:59:49.990243 12992 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 98d1bb7030104c4da7fcd66a495fdf43: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260812 06:59:50.095129 13011 catalog_manager.cc:1259] Loaded cluster ID: 18cdd7e4effe488d960bb9b3ccb8c8a5
I20260812 06:59:50.095240 13011 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 8859fc8b085a42ccbbc935c4dfaa4211: loading cluster ID for follower catalog manager: success
I20260812 06:59:50.097013 13011 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 8859fc8b085a42ccbbc935c4dfaa4211: acquiring CA information for follower catalog manager: success
I20260812 06:59:50.098203 13011 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 8859fc8b085a42ccbbc935c4dfaa4211: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260812 06:59:50.283977 13118 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:59:50.280090 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60818 (local address 127.19.52.125:42015)
0812 06:59:50.280354 (+ 264us) server_negotiation.cc:206] Beginning negotiation
0812 06:59:50.280358 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 06:59:50.280397 (+ 39us) server_negotiation.cc:407] Connection header received
0812 06:59:50.280451 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:59:50.280453 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:59:50.280618 (+ 165us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:59:50.280709 (+ 91us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 06:59:50.280998 (+ 289us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:59:50.281591 (+ 593us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:59:50.282443 (+ 852us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:59:50.282675 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:59:50.282750 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:59:50.283052 (+ 302us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 06:59:50.283488 (+ 436us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 06:59:50.283598 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0812 06:59:50.283701 (+ 103us) server_negotiation.cc:299] Negotiation successful
0812 06:59:50.283757 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":191,"thread_start_us":102,"threads_started":1}
I20260812 06:59:50.291852 13124 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:59:50.287436 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33480 (local address 127.19.52.124:33027)
0812 06:59:50.287702 (+ 266us) server_negotiation.cc:206] Beginning negotiation
0812 06:59:50.287706 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 06:59:50.287721 (+ 15us) server_negotiation.cc:407] Connection header received
0812 06:59:50.287831 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:59:50.287833 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:59:50.287949 (+ 116us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:59:50.288110 (+ 161us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 06:59:50.288373 (+ 263us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:59:50.288973 (+ 600us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:59:50.290015 (+ 1042us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:59:50.290244 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:59:50.290415 (+ 171us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:59:50.290762 (+ 347us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 06:59:50.291296 (+ 534us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 06:59:50.291394 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0812 06:59:50.291567 (+ 173us) server_negotiation.cc:299] Negotiation successful
0812 06:59:50.291668 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":181,"thread_start_us":81,"threads_started":1}
I20260812 06:59:50.299795 13124 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:59:50.295857 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33486 (local address 127.19.52.124:33027)
0812 06:59:50.296101 (+ 244us) server_negotiation.cc:206] Beginning negotiation
0812 06:59:50.296104 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 06:59:50.296114 (+ 10us) server_negotiation.cc:407] Connection header received
0812 06:59:50.296153 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:59:50.296155 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:59:50.296245 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:59:50.296323 (+ 78us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 06:59:50.296691 (+ 368us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:59:50.297504 (+ 813us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:59:50.298463 (+ 959us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:59:50.298817 (+ 354us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:59:50.298920 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:59:50.299282 (+ 362us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 06:59:50.299371 (+ 89us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 06:59:50.299456 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0812 06:59:50.299583 (+ 127us) server_negotiation.cc:299] Negotiation successful
0812 06:59:50.299642 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":182}
I20260812 06:59:50.300490 13131 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:59:50.295643 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35412 (local address 127.19.52.126:42867)
0812 06:59:50.295910 (+ 267us) server_negotiation.cc:206] Beginning negotiation
0812 06:59:50.295915 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 06:59:50.295929 (+ 14us) server_negotiation.cc:407] Connection header received
0812 06:59:50.296022 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:59:50.296024 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:59:50.296263 (+ 239us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:59:50.296392 (+ 129us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 06:59:50.296763 (+ 371us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:59:50.297535 (+ 772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:59:50.298621 (+ 1086us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:59:50.298851 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:59:50.298967 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:59:50.299394 (+ 427us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 06:59:50.299766 (+ 372us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 06:59:50.299857 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0812 06:59:50.300111 (+ 254us) server_negotiation.cc:299] Negotiation successful
0812 06:59:50.300306 (+ 195us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":176,"thread_start_us":73,"threads_started":1}
I20260812 06:59:50.300730 13118 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:59:50.295489 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60822 (local address 127.19.52.125:42015)
0812 06:59:50.296023 (+ 534us) server_negotiation.cc:206] Beginning negotiation
0812 06:59:50.296026 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 06:59:50.296883 (+ 857us) server_negotiation.cc:407] Connection header received
0812 06:59:50.296984 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:59:50.296987 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:59:50.297047 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:59:50.297123 (+ 76us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 06:59:50.297377 (+ 254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:59:50.298134 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:59:50.299235 (+ 1101us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:59:50.299537 (+ 302us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:59:50.299643 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:59:50.300193 (+ 550us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 06:59:50.300286 (+ 93us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 06:59:50.300336 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0812 06:59:50.300520 (+ 184us) server_negotiation.cc:299] Negotiation successful
0812 06:59:50.300611 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":425}
I20260812 06:59:50.304380 19665 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu with pid 12724
I20260812 06:59:50.315734 19665 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu with pid 12810
I20260812 06:59:50.324003 19665 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu with pid 12894
2026-08-12T06:59:50Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0 (3407 ms)
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1
Loading random data
Initializing database '/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786517687384214-19665-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 06:59:50 dist-test-slave-65mx krb5kdc[13135](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 06:59:50 dist-test-slave-65mx krb5kdc[13135](info): set up 2 sockets
Aug 12 06:59:50 dist-test-slave-65mx krb5kdc[13135](info): commencing operation
krb5kdc: starting...
W20260812 06:59:51.451494 19665 mini_kdc.cc:121] Time spent starting KDC: real 1.101s user 0.008s sys 0.000s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786517687384214-19665-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786517687384214-19665-0/krb5kdc/test-user.keytab.
Aug 12 06:59:51 dist-test-slave-65mx krb5kdc[13135](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517991, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T06:59:51Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T06:59:51Z Disabled control of system clock
WARNING: no policy specified for kudu/127.19.52.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.52.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.52.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786517687384214-19665-0/krb5kdc/kudu_127.19.52.126.keytab.
Entry for principal kudu/127.19.52.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786517687384214-19665-0/krb5kdc/kudu_127.19.52.126.keytab.
I20260812 06:59:51.596421 19665 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu
/tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786517687384214-19665-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786517687384214-19665-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786517687384214-19665-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786517687384214-19665-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.52.126:34591
--webserver_interface=127.19.52.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786517687384214-19665-0/krb5kdc/kudu_127.19.52.126.keytab
--principal=kudu/127.19.52.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.52.84:43495
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.19.52.126:34591,127.19.52.125:46687,127.19.52.124:44467
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786517687384214-19665-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786517687384214-19665-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786517687384214-19665-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 06:59:51.723363 13153 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260812 06:59:51.723737 13153 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:59:51.723804 13153 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:59:51.723886 13153 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 06:59:51.728057 13153 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 06:59:51.728168 13153 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 06:59:51.728207 13153 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 06:59:51.728261 13153 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 06:59:51.728300 13153 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 06:59:51.733852 13153 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.52.84:43495
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786517687384214-19665-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786517687384214-19665-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.19.52.126:34591,127.19.52.125:46687,127.19.52.124:44467
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786517687384214-19665-0/krb5kdc/kudu_127.19.52.126.keytab
--principal=kudu/127.19.52.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.52.126:34591
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786517687384214-19665-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.19.52.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13153
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786517687384214-19665-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 06:15:54 UTC on ccycloud.quasar-rnfzce.root.comops.site
build id 4581989
I20260812 06:59:51.735037 13153 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:59:51.736107 13153 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 06:59:51.744612 13161 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:59:51.744709 13158 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:59:51.744894 13153 server_base.cc:1034] running on GCE node
W20260812 06:59:51.744720 13159 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:59:51.745612 13153 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:59:51.746822 13153 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:59:51.748044 13153 hybrid_clock.cc:648] HybridClock initialized: now 1786517991747991 us; error 57 us; skew 500 ppm
Aug 12 06:59:51 dist-test-slave-65mx krb5kdc[13135](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517991, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.52.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 06:59:51.751366 13153 init.cc:377] Logged in from keytab as kudu/127.19.52.126@KRBTEST.COM (short username kudu)
I20260812 06:59:51.752420 13153 webserver.cc:460] Webserver started at http://127.19.52.126:32957/ using document root <none> and password file <none>
I20260812 06:59:51.753072 13153 fs_manager.cc:359] Metadata directory not provided
I20260812 06:59:51.753175 13153 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:59:51.753424 13153 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 06:59:51.755848 13153 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786517687384214-19665-0/minicluster-data/master-0/data/instance:
uuid: "67fab72295ac409dabb8e3e247d084f1"
format_stamp: "Formatted at 2026-08-12 06:59:51 on dist-test-slave-65mx"
server_key: "e3e178f0920cb687a417e14e32a0acb0"
server_key_iv: "17514d1758c9cc7baa3774d7cbd916b0"
server_key_version: "encryptionkey@0"
I20260812 06:59:51.756511 13153 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786517687384214-19665-0/minicluster-data/master-0/wal/instance:
uuid: "67fab72295ac409dabb8e3e247d084f1"
format_stamp: "Formatted at 2026-08-12 06:59:51 on dist-test-slave-65mx"
server_key: "e3e178f0920cb687a417e14e32a0acb0"
server_key_iv: "17514d1758c9cc7baa3774d7cbd916b0"
server_key_version: "encryptionkey@0"
I20260812 06:59:51.760486 13153 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260812 06:59:51.763013 13171 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 06:59:51.764400 13153 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260812 06:59:51.764631 13153 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786517687384214-19665-0/minicluster-data/master-0/data,/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786517687384214-19665-0/minicluster-data/master-0/wal
uuid: "67fab72295ac409dabb8e3e247d084f1"
format_stamp: "Formatted at 2026-08-12 06:59:51 on dist-test-slave-65mx"
server_key: "e3e178f0920cb687a417e14e32a0acb0"
server_key_iv: "17514d1758c9cc7baa3774d7cbd916b0"
server_key_version: "encryptionkey@0"
I20260812 06:59:51.764771 13153 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786517687384214-19665-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786517687384214-19665-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786517687384214-19665-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:59:51.788312 13153 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 06:59:51.791760 13153 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 06:59:51.792011 13153 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:59:51.800355 13153 rpc_server.cc:307] RPC server started. Bound to: 127.19.52.126:34591
I20260812 06:59:51.800695 13227 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.52.126:34591 every 8 connection(s)
I20260812 06:59:51.801503 13153 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786517687384214-19665-0/minicluster-data/master-0/data/info.pb
I20260812 06:59:51.804313 13230 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 06:59:51.805343 19665 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu as pid 13153
I20260812 06:59:51.805483 19665 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786517687384214-19665-0/minicluster-data/master-0/wal/instance
I20260812 06:59:51.805809 19665 external_mini_cluster.cc:1428] Setting key c9cb52dab8269cad8e3dcb64188a869a
I20260812 06:59:51.809731 13230 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.19.52.126" port: 34591 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.19.52.125@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.52.125@KRBTEST.COM" created.
Aug 12 06:59:51 dist-test-slave-65mx krb5kdc[13135](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517991, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.52.126@KRBTEST.COM for kudu/127.19.52.126@KRBTEST.COM
I20260812 06:59:51.826669 13233 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:59:51.812429 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47916 (local address 127.19.52.126:34591)
0812 06:59:51.812980 (+ 551us) server_negotiation.cc:206] Beginning negotiation
0812 06:59:51.813002 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0812 06:59:51.813297 (+ 295us) server_negotiation.cc:407] Connection header received
0812 06:59:51.814560 (+ 1263us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:59:51.814565 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:59:51.814850 (+ 285us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:59:51.815056 (+ 206us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:59:51.816035 (+ 979us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:59:51.817066 (+ 1031us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:59:51.818082 (+ 1016us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:59:51.818481 (+ 399us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:59:51.818661 (+ 180us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:59:51.821266 (+ 2605us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:59:51.821272 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:59:51.821282 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:59:51.821304 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:59:51.823198 (+ 1894us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:59:51.823747 (+ 549us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:59:51.823753 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:59:51.823755 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:59:51.823867 (+ 112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:59:51.824191 (+ 324us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:59:51.824197 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:59:51.824199 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:59:51.824682 (+ 483us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:59:51.824822 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0812 06:59:51.825248 (+ 426us) server_negotiation.cc:299] Negotiation successful
0812 06:59:51.825534 (+ 286us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":379,"thread_start_us":157,"threads_started":1}
I20260812 06:59:51.826669 13234 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:59:51.812511 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.126:34591 (local address 127.0.0.1:47916)
0812 06:59:51.813139 (+ 628us) negotiation.cc:107] Waiting for socket to connect
0812 06:59:51.813180 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0812 06:59:51.814218 (+ 1038us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:59:51.815056 (+ 838us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:59:51.815061 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:59:51.815555 (+ 494us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:59:51.815875 (+ 320us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:59:51.815891 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:59:51.817193 (+ 1302us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:59:51.817197 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:59:51.817950 (+ 753us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:59:51.817960 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:59:51.818653 (+ 693us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:59:51.818656 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:59:51.818782 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:59:51.819477 (+ 695us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 06:59:51.819495 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:59:51.821080 (+ 1585us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 06:59:51.823352 (+ 2272us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:59:51.823363 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:59:51.823373 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:59:51.823632 (+ 259us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:59:51.823966 (+ 334us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:59:51.823971 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:59:51.823973 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:59:51.824099 (+ 126us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:59:51.824802 (+ 703us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 06:59:51.824809 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 06:59:51.824957 (+ 148us) client_negotiation.cc:769] Sending connection context
0812 06:59:51.825184 (+ 227us) client_negotiation.cc:240] Negotiation successful
0812 06:59:51.825581 (+ 397us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":454,"mutex_wait_us":111,"thread_start_us":211,"threads_started":1}
I20260812 06:59:51.829207 13230 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.19.52.125" port: 46687 } has no permanent_uuid. Determining permanent_uuid...
I20260812 06:59:51.830261 13234 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:59:51.829753 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.125:46687 (local address 127.0.0.1:55942)
0812 06:59:51.829901 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0812 06:59:51.829986 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.52.125:46687: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
W20260812 06:59:51.830543 13172 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.19.52.125:46687: connect: Connection refused (error 111)
I20260812 06:59:51.831586 13234 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:59:51.831250 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.125:46687 (local address 127.0.0.1:55948)
0812 06:59:51.831472 (+ 222us) negotiation.cc:107] Waiting for socket to connect
0812 06:59:51.831524 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.52.125:46687: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":77}
W20260812 06:59:51.831830 13230 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.19.52.125:46687: Network error: Client connection negotiation failed: client connection to 127.19.52.125:46687: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.52.125 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786517687384214-19665-0/krb5kdc/kudu_127.19.52.125.keytab.
Entry for principal kudu/127.19.52.125 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786517687384214-19665-0/krb5kdc/kudu_127.19.52.125.keytab.
I20260812 06:59:51.840265 19665 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu
/tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786517687384214-19665-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786517687384214-19665-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786517687384214-19665-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786517687384214-19665-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.52.125:46687
--webserver_interface=127.19.52.125
--webserver_port=0
--keytab_file=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786517687384214-19665-0/krb5kdc/kudu_127.19.52.125.keytab
--principal=kudu/127.19.52.125
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.52.84:43495
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.19.52.126:34591,127.19.52.125:46687,127.19.52.124:44467
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786517687384214-19665-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786517687384214-19665-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786517687384214-19665-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 06:59:51.881134 13230 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.19.52.125" port: 46687 } attempt: 1
I20260812 06:59:51.882701 13234 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:59:51.882336 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.125:46687 (local address 127.0.0.1:55954)
0812 06:59:51.882543 (+ 207us) negotiation.cc:107] Waiting for socket to connect
0812 06:59:51.882612 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.52.125:46687: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":96}
I20260812 06:59:51.883560 13234 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:59:51.883314 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.125:46687 (local address 127.0.0.1:55966)
0812 06:59:51.883436 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0812 06:59:51.883494 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.52.125:46687: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260812 06:59:51.883708 13230 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.19.52.125:46687: Network error: Client connection negotiation failed: client connection to 127.19.52.125:46687: connect: Connection refused (error 111)
I20260812 06:59:51.951895 13230 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.19.52.125" port: 46687 } attempt: 2
I20260812 06:59:51.952842 13234 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:59:51.952483 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.125:46687 (local address 127.0.0.1:55970)
0812 06:59:51.952695 (+ 212us) negotiation.cc:107] Waiting for socket to connect
0812 06:59:51.952758 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.52.125:46687: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":104}
I20260812 06:59:51.953593 13234 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:59:51.953372 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.125:46687 (local address 127.0.0.1:55974)
0812 06:59:51.953479 (+ 107us) negotiation.cc:107] Waiting for socket to connect
0812 06:59:51.953537 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.52.125:46687: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
W20260812 06:59:51.953729 13230 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.19.52.125:46687: Network error: Client connection negotiation failed: client connection to 127.19.52.125:46687: connect: Connection refused (error 111)
W20260812 06:59:51.963078 13238 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260812 06:59:51.963449 13238 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:59:51.963515 13238 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:59:51.963593 13238 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 06:59:51.967789 13238 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 06:59:51.967885 13238 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 06:59:51.967923 13238 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 06:59:51.967978 13238 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 06:59:51.968015 13238 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 06:59:51.973726 13238 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.52.84:43495
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786517687384214-19665-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786517687384214-19665-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.19.52.126:34591,127.19.52.125:46687,127.19.52.124:44467
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786517687384214-19665-0/krb5kdc/kudu_127.19.52.125.keytab
--principal=kudu/127.19.52.125
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.52.125:46687
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786517687384214-19665-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.19.52.125
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13238
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786517687384214-19665-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 06:15:54 UTC on ccycloud.quasar-rnfzce.root.comops.site
build id 4581989
I20260812 06:59:51.974975 13238 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:59:51.976076 13238 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 06:59:51.984944 13244 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:59:51.984947 13243 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:59:51.984954 13246 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:59:51.985307 13238 server_base.cc:1034] running on GCE node
I20260812 06:59:51.985970 13238 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:59:51.987123 13238 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:59:51.988421 13238 hybrid_clock.cc:648] HybridClock initialized: now 1786517991988404 us; error 32 us; skew 500 ppm
Aug 12 06:59:51 dist-test-slave-65mx krb5kdc[13135](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517991, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.52.125@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 06:59:51.991384 13238 init.cc:377] Logged in from keytab as kudu/127.19.52.125@KRBTEST.COM (short username kudu)
I20260812 06:59:51.992303 13238 webserver.cc:460] Webserver started at http://127.19.52.125:37323/ using document root <none> and password file <none>
I20260812 06:59:51.992941 13238 fs_manager.cc:359] Metadata directory not provided
I20260812 06:59:51.993036 13238 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:59:51.993275 13238 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 06:59:51.995343 13238 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786517687384214-19665-0/minicluster-data/master-1/data/instance:
uuid: "b6886ade627940d896a39d91e0348019"
format_stamp: "Formatted at 2026-08-12 06:59:51 on dist-test-slave-65mx"
server_key: "ceb77527d06c4b220b9c0015af13c627"
server_key_iv: "c4d4d3d0ed822f886e326f2541683672"
server_key_version: "encryptionkey@0"
I20260812 06:59:51.995920 13238 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786517687384214-19665-0/minicluster-data/master-1/wal/instance:
uuid: "b6886ade627940d896a39d91e0348019"
format_stamp: "Formatted at 2026-08-12 06:59:51 on dist-test-slave-65mx"
server_key: "ceb77527d06c4b220b9c0015af13c627"
server_key_iv: "c4d4d3d0ed822f886e326f2541683672"
server_key_version: "encryptionkey@0"
I20260812 06:59:51.999848 13238 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260812 06:59:52.002398 13254 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 06:59:52.003757 13238 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260812 06:59:52.003916 13238 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786517687384214-19665-0/minicluster-data/master-1/data,/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786517687384214-19665-0/minicluster-data/master-1/wal
uuid: "b6886ade627940d896a39d91e0348019"
format_stamp: "Formatted at 2026-08-12 06:59:51 on dist-test-slave-65mx"
server_key: "ceb77527d06c4b220b9c0015af13c627"
server_key_iv: "c4d4d3d0ed822f886e326f2541683672"
server_key_version: "encryptionkey@0"
I20260812 06:59:52.004031 13238 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786517687384214-19665-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786517687384214-19665-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786517687384214-19665-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:59:52.040774 13238 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 06:59:52.044116 13238 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 06:59:52.044345 13238 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:59:52.044901 13230 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.19.52.125" port: 46687 } attempt: 3
I20260812 06:59:52.045753 13234 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:59:52.045391 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.125:46687 (local address 127.0.0.1:55984)
0812 06:59:52.045581 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0812 06:59:52.045653 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.52.125:46687: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
I20260812 06:59:52.046540 13234 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:59:52.046319 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.125:46687 (local address 127.0.0.1:55992)
0812 06:59:52.046430 (+ 111us) negotiation.cc:107] Waiting for socket to connect
0812 06:59:52.046476 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.52.125:46687: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260812 06:59:52.046761 13230 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.19.52.125:46687: Network error: Client connection negotiation failed: client connection to 127.19.52.125:46687: connect: Connection refused (error 111)
I20260812 06:59:52.052325 13238 rpc_server.cc:307] RPC server started. Bound to: 127.19.52.125:46687
I20260812 06:59:52.052690 13311 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.52.125:46687 every 8 connection(s)
I20260812 06:59:52.053385 13238 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786517687384214-19665-0/minicluster-data/master-1/data/info.pb
I20260812 06:59:52.056123 13312 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 06:59:52.058734 19665 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu as pid 13238
I20260812 06:59:52.058917 19665 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786517687384214-19665-0/minicluster-data/master-1/wal/instance
I20260812 06:59:52.059216 19665 external_mini_cluster.cc:1428] Setting key e49d5f0dfa46610821b62a3f8539ec0d
I20260812 06:59:52.059935 13312 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.19.52.126" port: 34591 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.19.52.124@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.52.124@KRBTEST.COM" created.
Aug 12 06:59:52 dist-test-slave-65mx krb5kdc[13135](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517991, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.52.125@KRBTEST.COM for kudu/127.19.52.126@KRBTEST.COM
I20260812 06:59:52.078029 13233 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:59:52.061940 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47930 (local address 127.19.52.126:34591)
0812 06:59:52.062115 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0812 06:59:52.062121 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 06:59:52.062927 (+ 806us) server_negotiation.cc:407] Connection header received
0812 06:59:52.063882 (+ 955us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:59:52.063888 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:59:52.063963 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:59:52.064098 (+ 135us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:59:52.065361 (+ 1263us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:59:52.066270 (+ 909us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:59:52.067421 (+ 1151us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:59:52.067835 (+ 414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:59:52.067968 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:59:52.073098 (+ 5130us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:59:52.073105 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:59:52.073111 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:59:52.073148 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:59:52.075422 (+ 2274us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:59:52.076127 (+ 705us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:59:52.076134 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:59:52.076140 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:59:52.076236 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:59:52.076683 (+ 447us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:59:52.076687 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:59:52.076696 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:59:52.076920 (+ 224us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:59:52.077071 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0812 06:59:52.077667 (+ 596us) server_negotiation.cc:299] Negotiation successful
0812 06:59:52.077825 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260812 06:59:52.078930 13315 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:59:52.062274 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.126:34591 (local address 127.0.0.1:47930)
0812 06:59:52.062757 (+ 483us) negotiation.cc:107] Waiting for socket to connect
0812 06:59:52.062809 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0812 06:59:52.063668 (+ 859us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:59:52.064291 (+ 623us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:59:52.064298 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:59:52.064718 (+ 420us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:59:52.065198 (+ 480us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:59:52.065211 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:59:52.066408 (+ 1197us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:59:52.066411 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:59:52.067290 (+ 879us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:59:52.067299 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:59:52.067995 (+ 696us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:59:52.067998 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:59:52.068145 (+ 147us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:59:52.068865 (+ 720us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 06:59:52.068882 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:59:52.072888 (+ 4006us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 06:59:52.075588 (+ 2700us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:59:52.075594 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:59:52.075651 (+ 57us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:59:52.075964 (+ 313us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:59:52.076352 (+ 388us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:59:52.076357 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:59:52.076359 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:59:52.076524 (+ 165us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:59:52.077061 (+ 537us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 06:59:52.077069 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 06:59:52.077363 (+ 294us) client_negotiation.cc:769] Sending connection context
0812 06:59:52.077585 (+ 222us) client_negotiation.cc:240] Negotiation successful
0812 06:59:52.077962 (+ 377us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":287,"thread_start_us":151,"threads_started":1}
I20260812 06:59:52.080482 13312 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.19.52.125" port: 46687 } has no permanent_uuid. Determining permanent_uuid...
Aug 12 06:59:52 dist-test-slave-65mx krb5kdc[13135](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517991, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.52.125@KRBTEST.COM for kudu/127.19.52.125@KRBTEST.COM
I20260812 06:59:52.090622 13315 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:59:52.081076 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.125:46687 (local address 127.0.0.1:56004)
0812 06:59:52.081199 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0812 06:59:52.081214 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 06:59:52.081293 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:59:52.082051 (+ 758us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:59:52.082055 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:59:52.082199 (+ 144us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:59:52.082296 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:59:52.082301 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:59:52.083426 (+ 1125us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:59:52.083429 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:59:52.083954 (+ 525us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:59:52.083962 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:59:52.084679 (+ 717us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:59:52.084682 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:59:52.084757 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:59:52.085214 (+ 457us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 06:59:52.085225 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:59:52.086543 (+ 1318us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 06:59:52.088889 (+ 2346us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:59:52.088892 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:59:52.088893 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:59:52.089211 (+ 318us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:59:52.089526 (+ 315us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:59:52.089528 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:59:52.089529 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:59:52.089665 (+ 136us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:59:52.090178 (+ 513us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 06:59:52.090180 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 06:59:52.090261 (+ 81us) client_negotiation.cc:769] Sending connection context
0812 06:59:52.090355 (+ 94us) client_negotiation.cc:240] Negotiation successful
0812 06:59:52.090477 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":48}
I20260812 06:59:52.090885 13317 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:59:52.081231 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56004 (local address 127.19.52.125:46687)
0812 06:59:52.081611 (+ 380us) server_negotiation.cc:206] Beginning negotiation
0812 06:59:52.081621 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0812 06:59:52.081642 (+ 21us) server_negotiation.cc:407] Connection header received
0812 06:59:52.081685 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:59:52.081689 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:59:52.081835 (+ 146us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:59:52.082060 (+ 225us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:59:52.082395 (+ 335us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:59:52.083290 (+ 895us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:59:52.084119 (+ 829us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:59:52.084511 (+ 392us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:59:52.084659 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:59:52.086719 (+ 2060us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:59:52.086731 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:59:52.086748 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:59:52.086780 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:59:52.088755 (+ 1975us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:59:52.089327 (+ 572us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:59:52.089332 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:59:52.089333 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:59:52.089414 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:59:52.089757 (+ 343us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:59:52.089760 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:59:52.089761 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:59:52.090061 (+ 300us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:59:52.090173 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0812 06:59:52.090484 (+ 311us) server_negotiation.cc:299] Negotiation successful
0812 06:59:52.090616 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":275,"thread_start_us":158,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.52.124 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786517687384214-19665-0/krb5kdc/kudu_127.19.52.124.keytab.
Entry for principal kudu/127.19.52.124 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786517687384214-19665-0/krb5kdc/kudu_127.19.52.124.keytab.
I20260812 06:59:52.092113 19665 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu
/tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786517687384214-19665-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786517687384214-19665-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786517687384214-19665-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786517687384214-19665-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.52.124:44467
--webserver_interface=127.19.52.124
--webserver_port=0
--keytab_file=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786517687384214-19665-0/krb5kdc/kudu_127.19.52.124.keytab
--principal=kudu/127.19.52.124
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.52.84:43495
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.19.52.126:34591,127.19.52.125:46687,127.19.52.124:44467
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786517687384214-19665-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786517687384214-19665-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786517687384214-19665-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 06:59:52.092917 13312 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.19.52.124" port: 44467 } has no permanent_uuid. Determining permanent_uuid...
I20260812 06:59:52.093732 13315 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:59:52.093399 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.124:44467 (local address 127.0.0.1:34332)
0812 06:59:52.093545 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0812 06:59:52.093642 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.52.124:44467: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
W20260812 06:59:52.093981 13260 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.19.52.124:44467: connect: Connection refused (error 111)
I20260812 06:59:52.094978 13315 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:59:52.094707 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.124:44467 (local address 127.0.0.1:34336)
0812 06:59:52.094830 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0812 06:59:52.094878 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.52.124:44467: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
W20260812 06:59:52.095265 13312 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.19.52.124:44467: Network error: Client connection negotiation failed: client connection to 127.19.52.124:44467: connect: Connection refused (error 111)
I20260812 06:59:52.144626 13312 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.19.52.124" port: 44467 } attempt: 1
I20260812 06:59:52.145664 13315 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:59:52.145287 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.124:44467 (local address 127.0.0.1:34352)
0812 06:59:52.145488 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0812 06:59:52.145571 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.52.124:44467: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":80}
I20260812 06:59:52.146517 13315 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:59:52.146274 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.124:44467 (local address 127.0.0.1:34362)
0812 06:59:52.146399 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0812 06:59:52.146451 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.52.124:44467: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
W20260812 06:59:52.146692 13312 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.19.52.124:44467: Network error: Client connection negotiation failed: client connection to 127.19.52.124:44467: connect: Connection refused (error 111)
I20260812 06:59:52.189937 13230 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.19.52.125" port: 46687 } attempt: 4
Aug 12 06:59:52 dist-test-slave-65mx krb5kdc[13135](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517991, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.52.126@KRBTEST.COM for kudu/127.19.52.125@KRBTEST.COM
I20260812 06:59:52.201515 13234 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:59:52.190600 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.125:46687 (local address 127.0.0.1:56020)
0812 06:59:52.190788 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0812 06:59:52.190806 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0812 06:59:52.190911 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:59:52.191268 (+ 357us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:59:52.191287 (+ 19us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:59:52.191511 (+ 224us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:59:52.191664 (+ 153us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:59:52.191672 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:59:52.193225 (+ 1553us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:59:52.193230 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:59:52.194229 (+ 999us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:59:52.194247 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:59:52.194896 (+ 649us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:59:52.194901 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:59:52.195057 (+ 156us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:59:52.195683 (+ 626us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 06:59:52.195704 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:59:52.197412 (+ 1708us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 06:59:52.199611 (+ 2199us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:59:52.199614 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:59:52.199616 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:59:52.199918 (+ 302us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:59:52.200295 (+ 377us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:59:52.200298 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:59:52.200300 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:59:52.200393 (+ 93us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:59:52.200903 (+ 510us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 06:59:52.200908 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 06:59:52.201014 (+ 106us) client_negotiation.cc:769] Sending connection context
0812 06:59:52.201103 (+ 89us) client_negotiation.cc:240] Negotiation successful
0812 06:59:52.201265 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":96}
I20260812 06:59:52.202250 13317 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:59:52.190685 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56020 (local address 127.19.52.125:46687)
0812 06:59:52.190839 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0812 06:59:52.190844 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 06:59:52.190866 (+ 22us) server_negotiation.cc:407] Connection header received
0812 06:59:52.191029 (+ 163us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:59:52.191032 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:59:52.191086 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:59:52.191235 (+ 149us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:59:52.191799 (+ 564us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:59:52.193101 (+ 1302us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:59:52.194388 (+ 1287us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:59:52.194760 (+ 372us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:59:52.194872 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:59:52.197587 (+ 2715us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:59:52.197591 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:59:52.197593 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:59:52.197637 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:59:52.199448 (+ 1811us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:59:52.200080 (+ 632us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:59:52.200086 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:59:52.200089 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:59:52.200195 (+ 106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:59:52.200504 (+ 309us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:59:52.200509 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:59:52.200511 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:59:52.200793 (+ 282us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:59:52.201762 (+ 969us) server_negotiation.cc:1035] Waiting for connection context
0812 06:59:52.201850 (+ 88us) server_negotiation.cc:299] Negotiation successful
0812 06:59:52.202003 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260812 06:59:52.203027 13230 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.19.52.124" port: 44467 } has no permanent_uuid. Determining permanent_uuid...
I20260812 06:59:52.203836 13234 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:59:52.203589 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.124:44467 (local address 127.0.0.1:34374)
0812 06:59:52.203716 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0812 06:59:52.203768 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.52.124:44467: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
I20260812 06:59:52.204531 13234 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:59:52.204298 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.124:44467 (local address 127.0.0.1:34378)
0812 06:59:52.204407 (+ 109us) negotiation.cc:107] Waiting for socket to connect
0812 06:59:52.204461 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.52.124:44467: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260812 06:59:52.204715 13230 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.19.52.124:44467: Network error: Client connection negotiation failed: client connection to 127.19.52.124:44467: connect: Connection refused (error 111)
I20260812 06:59:52.214895 13312 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.19.52.124" port: 44467 } attempt: 2
I20260812 06:59:52.215829 13315 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:59:52.215477 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.124:44467 (local address 127.0.0.1:34394)
0812 06:59:52.215666 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0812 06:59:52.215734 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.52.124:44467: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":77}
I20260812 06:59:52.216812 13315 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:59:52.216436 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.124:44467 (local address 127.0.0.1:34402)
0812 06:59:52.216653 (+ 217us) negotiation.cc:107] Waiting for socket to connect
0812 06:59:52.216718 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.52.124:44467: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":129}
W20260812 06:59:52.217056 13312 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.19.52.124:44467: Network error: Client connection negotiation failed: client connection to 127.19.52.124:44467: connect: Connection refused (error 111)
W20260812 06:59:52.231786 13318 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260812 06:59:52.232291 13318 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:59:52.232357 13318 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:59:52.232443 13318 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 06:59:52.239711 13318 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 06:59:52.239792 13318 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 06:59:52.239812 13318 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 06:59:52.239830 13318 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 06:59:52.239846 13318 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 06:59:52.245615 13318 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.52.84:43495
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786517687384214-19665-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786517687384214-19665-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.19.52.126:34591,127.19.52.125:46687,127.19.52.124:44467
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786517687384214-19665-0/krb5kdc/kudu_127.19.52.124.keytab
--principal=kudu/127.19.52.124
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.52.124:44467
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786517687384214-19665-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.19.52.124
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13318
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786517687384214-19665-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 06:15:54 UTC on ccycloud.quasar-rnfzce.root.comops.site
build id 4581989
I20260812 06:59:52.246763 13318 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:59:52.247833 13318 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 06:59:52.256528 13325 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:59:52.256619 13327 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:59:52.256583 13318 server_base.cc:1034] running on GCE node
W20260812 06:59:52.256524 13324 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:59:52.257400 13318 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:59:52.258648 13318 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:59:52.259866 13318 hybrid_clock.cc:648] HybridClock initialized: now 1786517992259833 us; error 57 us; skew 500 ppm
Aug 12 06:59:52 dist-test-slave-65mx krb5kdc[13135](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517992, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.52.124@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 06:59:52.262740 13318 init.cc:377] Logged in from keytab as kudu/127.19.52.124@KRBTEST.COM (short username kudu)
I20260812 06:59:52.263687 13318 webserver.cc:460] Webserver started at http://127.19.52.124:46881/ using document root <none> and password file <none>
I20260812 06:59:52.263876 13230 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.19.52.124" port: 44467 } attempt: 1
I20260812 06:59:52.264307 13318 fs_manager.cc:359] Metadata directory not provided
I20260812 06:59:52.264375 13318 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:59:52.264652 13318 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 06:59:52.264819 13234 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:59:52.264432 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.124:44467 (local address 127.0.0.1:34406)
0812 06:59:52.264639 (+ 207us) negotiation.cc:107] Waiting for socket to connect
0812 06:59:52.264705 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.52.124:44467: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
I20260812 06:59:52.265516 13234 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:59:52.265294 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.124:44467 (local address 127.0.0.1:34414)
0812 06:59:52.265405 (+ 111us) negotiation.cc:107] Waiting for socket to connect
0812 06:59:52.265454 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.52.124:44467: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
W20260812 06:59:52.265667 13230 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.19.52.124:44467: Network error: Client connection negotiation failed: client connection to 127.19.52.124:44467: connect: Connection refused (error 111)
I20260812 06:59:52.266942 13318 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786517687384214-19665-0/minicluster-data/master-2/data/instance:
uuid: "c3e9882221ea426292aa99f5a9053215"
format_stamp: "Formatted at 2026-08-12 06:59:52 on dist-test-slave-65mx"
server_key: "a0c226da993d5d4eab4b92d994362fc0"
server_key_iv: "9ba6b0a2c2b08934222855aca739949c"
server_key_version: "encryptionkey@0"
I20260812 06:59:52.267551 13318 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786517687384214-19665-0/minicluster-data/master-2/wal/instance:
uuid: "c3e9882221ea426292aa99f5a9053215"
format_stamp: "Formatted at 2026-08-12 06:59:52 on dist-test-slave-65mx"
server_key: "a0c226da993d5d4eab4b92d994362fc0"
server_key_iv: "9ba6b0a2c2b08934222855aca739949c"
server_key_version: "encryptionkey@0"
I20260812 06:59:52.271690 13318 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.004s
I20260812 06:59:52.274375 13339 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 06:59:52.275727 13318 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260812 06:59:52.275974 13318 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786517687384214-19665-0/minicluster-data/master-2/data,/tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786517687384214-19665-0/minicluster-data/master-2/wal
uuid: "c3e9882221ea426292aa99f5a9053215"
format_stamp: "Formatted at 2026-08-12 06:59:52 on dist-test-slave-65mx"
server_key: "a0c226da993d5d4eab4b92d994362fc0"
server_key_iv: "9ba6b0a2c2b08934222855aca739949c"
server_key_version: "encryptionkey@0"
I20260812 06:59:52.276109 13318 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786517687384214-19665-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786517687384214-19665-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786517687384214-19665-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:59:52.297859 13318 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 06:59:52.301362 13318 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 06:59:52.301599 13318 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:59:52.308245 13312 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.19.52.124" port: 44467 } attempt: 3
I20260812 06:59:52.309240 13315 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:59:52.308860 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.124:44467 (local address 127.0.0.1:34426)
0812 06:59:52.309039 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0812 06:59:52.309106 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.52.124:44467: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":77}
I20260812 06:59:52.309234 13318 rpc_server.cc:307] RPC server started. Bound to: 127.19.52.124:44467
I20260812 06:59:52.309546 13397 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.52.124:44467 every 8 connection(s)
I20260812 06:59:52.310446 13318 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786517687384214-19665-0/minicluster-data/master-2/data/info.pb
I20260812 06:59:52.311362 19665 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu as pid 13318
I20260812 06:59:52.311491 19665 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskG7Vh1a/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786517687384214-19665-0/minicluster-data/master-2/wal/instance
I20260812 06:59:52.311825 19665 external_mini_cluster.cc:1428] Setting key 8ae80cf0b31777648161b8f3be1c05ea
I20260812 06:59:52.314031 13398 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 12 06:59:52 dist-test-slave-65mx krb5kdc[13135](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517991, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.52.125@KRBTEST.COM for kudu/127.19.52.124@KRBTEST.COM
I20260812 06:59:52.319216 13398 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.19.52.126" port: 34591 } has no permanent_uuid. Determining permanent_uuid...
Aug 12 06:59:52 dist-test-slave-65mx krb5kdc[13135](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517991, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.19.52.124@KRBTEST.COM
I20260812 06:59:52.323176 13315 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:59:52.309853 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.124:44467 (local address 127.0.0.1:34436)
0812 06:59:52.309996 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0812 06:59:52.310010 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 06:59:52.310122 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:59:52.313020 (+ 2898us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:59:52.313023 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:59:52.313238 (+ 215us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:59:52.313360 (+ 122us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:59:52.313367 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:59:52.314866 (+ 1499us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:59:52.314870 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:59:52.315464 (+ 594us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:59:52.315478 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:59:52.316461 (+ 983us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:59:52.316465 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:59:52.316581 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:59:52.317188 (+ 607us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 06:59:52.317204 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:59:52.318750 (+ 1546us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 06:59:52.321220 (+ 2470us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:59:52.321226 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:59:52.321231 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:59:52.321502 (+ 271us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:59:52.321883 (+ 381us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:59:52.321888 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:59:52.321891 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:59:52.322013 (+ 122us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:59:52.322627 (+ 614us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 06:59:52.322630 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 06:59:52.322736 (+ 106us) client_negotiation.cc:769] Sending connection context
0812 06:59:52.322859 (+ 123us) client_negotiation.cc:240] Negotiation successful
0812 06:59:52.323008 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":78}
I20260812 06:59:52.324409 13399 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:59:52.310666 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34436 (local address 127.19.52.124:44467)
0812 06:59:52.311125 (+ 459us) server_negotiation.cc:206] Beginning negotiation
0812 06:59:52.311150 (+ 25us) server_negotiation.cc:399] Waiting for connection header
0812 06:59:52.311199 (+ 49us) server_negotiation.cc:407] Connection header received
0812 06:59:52.312170 (+ 971us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:59:52.312198 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:59:52.312677 (+ 479us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:59:52.313065 (+ 388us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:59:52.313506 (+ 441us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:59:52.314743 (+ 1237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:59:52.315617 (+ 874us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:59:52.316324 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:59:52.316489 (+ 165us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:59:52.318952 (+ 2463us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:59:52.318957 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:59:52.318970 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:59:52.318996 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:59:52.321082 (+ 2086us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:59:52.321642 (+ 560us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:59:52.321647 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:59:52.321652 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:59:52.321787 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:59:52.322131 (+ 344us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:59:52.322133 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:59:52.322135 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:59:52.322510 (+ 375us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:59:52.322659 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0812 06:59:52.323009 (+ 350us) server_negotiation.cc:299] Negotiation successful
0812 06:59:52.323570 (+ 561us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":274,"thread_start_us":143,"threads_started":1}
I20260812 06:59:52.324429 13402 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:59:52.312467 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34450 (local address 127.19.52.124:44467)
0812 06:59:52.314323 (+ 1856us) server_negotiation.cc:206] Beginning negotiation
0812 06:59:52.314331 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 06:59:52.314345 (+ 14us) server_negotiation.cc:407] Connection header received
0812 06:59:52.314377 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:59:52.314380 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:59:52.314418 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:59:52.314506 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:59:52.315437 (+ 931us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:59:52.316018 (+ 581us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:59:52.317136 (+ 1118us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:59:52.317481 (+ 345us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:59:52.317603 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:59:52.320531 (+ 2928us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:59:52.320535 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:59:52.320538 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:59:52.320598 (+ 60us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:59:52.322408 (+ 1810us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:59:52.322933 (+ 525us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:59:52.322939 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:59:52.322942 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:59:52.323050 (+ 108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:59:52.323379 (+ 329us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:59:52.323382 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:59:52.323384 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:59:52.323592 (+ 208us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:59:52.323726 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0812 06:59:52.324136 (+ 410us) server_negotiation.cc:299] Negotiation successful
0812 06:59:52.324248 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1787,"thread_start_us":167,"threads_started":1}
Aug 12 06:59:52 dist-test-slave-65mx krb5kdc[13135](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517992, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.52.124@KRBTEST.COM for kudu/127.19.52.126@KRBTEST.COM
I20260812 06:59:52.329077 13312 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b6886ade627940d896a39d91e0348019: Bootstrap starting.
I20260812 06:59:52.331634 13312 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b6886ade627940d896a39d91e0348019: Neither blocks nor log segments found. Creating new log.
I20260812 06:59:52.332593 13312 log.cc:824] T 00000000000000000000000000000000 P b6886ade627940d896a39d91e0348019: Log is configured to *not* fsync() on all Append() calls
I20260812 06:59:52.332808 13230 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.19.52.124" port: 44467 } attempt: 2
Aug 12 06:59:52 dist-test-slave-65mx krb5kdc[13135](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517991, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.19.52.126@KRBTEST.COM
I20260812 06:59:52.334506 13403 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:59:52.321399 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.126:34591 (local address 127.0.0.1:47942)
0812 06:59:52.322786 (+ 1387us) negotiation.cc:107] Waiting for socket to connect
0812 06:59:52.322844 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0812 06:59:52.323008 (+ 164us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:59:52.323441 (+ 433us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:59:52.323447 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:59:52.323916 (+ 469us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:59:52.324078 (+ 162us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:59:52.324087 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:59:52.325170 (+ 1083us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:59:52.325172 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:59:52.325997 (+ 825us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:59:52.326007 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:59:52.326495 (+ 488us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:59:52.326498 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:59:52.326604 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:59:52.327842 (+ 1238us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 06:59:52.327862 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:59:52.329471 (+ 1609us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 06:59:52.331705 (+ 2234us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:59:52.331716 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:59:52.331722 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:59:52.332038 (+ 316us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:59:52.332411 (+ 373us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:59:52.332417 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:59:52.332423 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:59:52.332520 (+ 97us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:59:52.333481 (+ 961us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 06:59:52.333491 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 06:59:52.333645 (+ 154us) client_negotiation.cc:769] Sending connection context
0812 06:59:52.334060 (+ 415us) client_negotiation.cc:240] Negotiation successful
0812 06:59:52.334277 (+ 217us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1199,"thread_start_us":114,"threads_started":1}
I20260812 06:59:52.335350 13312 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b6886ade627940d896a39d91e0348019: No bootstrap required, opened a new log
I20260812 06:59:52.336843 13233 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:59:52.321498 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47942 (local address 127.19.52.126:34591)
0812 06:59:52.321690 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0812 06:59:52.321697 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 06:59:52.322916 (+ 1219us) server_negotiation.cc:407] Connection header received
0812 06:59:52.323139 (+ 223us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:59:52.323142 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:59:52.323201 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:59:52.323261 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:59:52.324218 (+ 957us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:59:52.325049 (+ 831us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:59:52.326131 (+ 1082us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:59:52.326337 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:59:52.326475 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:59:52.329636 (+ 3161us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:59:52.329640 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:59:52.329643 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:59:52.329670 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:59:52.331316 (+ 1646us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:59:52.332168 (+ 852us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:59:52.332175 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:59:52.332180 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:59:52.332286 (+ 106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:59:52.332700 (+ 414us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:59:52.332706 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:59:52.332707 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:59:52.332958 (+ 251us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:59:52.333414 (+ 456us) server_negotiation.cc:1035] Waiting for connection context
0812 06:59:52.333914 (+ 500us) server_negotiation.cc:299] Negotiation successful
0812 06:59:52.336666 (+ 2752us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":88}
I20260812 06:59:52.337889 13398 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.19.52.125" port: 46687 } has no permanent_uuid. Determining permanent_uuid...
I20260812 06:59:52.338331 13405 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:59:52.326969 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47944 (local address 127.19.52.126:34591)
0812 06:59:52.327281 (+ 312us) server_negotiation.cc:206] Beginning negotiation
0812 06:59:52.327297 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0812 06:59:52.327311 (+ 14us) server_negotiation.cc:407] Connection header received
0812 06:59:52.327344 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:59:52.327346 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:59:52.327386 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:59:52.327475 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:59:52.328303 (+ 828us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:59:52.329091 (+ 788us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:59:52.330467 (+ 1376us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:59:52.330852 (+ 385us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:59:52.330973 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:59:52.334096 (+ 3123us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:59:52.334100 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:59:52.334102 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:59:52.334127 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:59:52.335900 (+ 1773us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:59:52.336347 (+ 447us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:59:52.336357 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:59:52.336359 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:59:52.336437 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:59:52.336771 (+ 334us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:59:52.336774 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:59:52.336776 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:59:52.337110 (+ 334us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:59:52.337310 (+ 200us) server_negotiation.cc:1035] Waiting for connection context
0812 06:59:52.337943 (+ 633us) server_negotiation.cc:299] Negotiation successful
0812 06:59:52.338098 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":237,"thread_start_us":80,"threads_started":1}
Aug 12 06:59:52 dist-test-slave-65mx krb5kdc[13135](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517991, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.52.126@KRBTEST.COM for kudu/127.19.52.124@KRBTEST.COM
I20260812 06:59:52.340197 13312 raft_consensus.cc:348] T 00000000000000000000000000000000 P b6886ade627940d896a39d91e0348019 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67fab72295ac409dabb8e3e247d084f1" member_type: VOTER last_known_addr { host: "127.19.52.126" port: 34591 } } peers { permanent_uuid: "b6886ade627940d896a39d91e0348019" member_type: VOTER last_known_addr { host: "127.19.52.125" port: 46687 } } peers { permanent_uuid: "c3e9882221ea426292aa99f5a9053215" member_type: VOTER last_known_addr { host: "127.19.52.124" port: 44467 } }
I20260812 06:59:52.340474 13312 raft_consensus.cc:729] T 00000000000000000000000000000000 P b6886ade627940d896a39d91e0348019 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b6886ade627940d896a39d91e0348019, State: Initialized, Role: FOLLOWER
I20260812 06:59:52.340914 13312 consensus_queue.cc:260] T 00000000000000000000000000000000 P b6886ade627940d896a39d91e0348019 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67fab72295ac409dabb8e3e247d084f1" member_type: VOTER last_known_addr { host: "127.19.52.126" port: 34591 } } peers { permanent_uuid: "b6886ade627940d896a39d91e0348019" member_type: VOTER last_known_addr { host: "127.19.52.125" port: 46687 } } peers { permanent_uuid: "c3e9882221ea426292aa99f5a9053215" member_type: VOTER last_known_addr { host: "127.19.52.124" port: 44467 } }
I20260812 06:59:52.341675 13409 sys_catalog.cc:455] T 00000000000000000000000000000000 P b6886ade627940d896a39d91e0348019 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67fab72295ac409dabb8e3e247d084f1" member_type: VOTER last_known_addr { host: "127.19.52.126" port: 34591 } } peers { permanent_uuid: "b6886ade627940d896a39d91e0348019" member_type: VOTER last_known_addr { host: "127.19.52.125" port: 46687 } } peers { permanent_uuid: "c3e9882221ea426292aa99f5a9053215" member_type: VOTER last_known_addr { host: "127.19.52.124" port: 44467 } } }
I20260812 06:59:52.341856 13409 sys_catalog.cc:458] T 00000000000000000000000000000000 P b6886ade627940d896a39d91e0348019 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 06:59:52.342767 13312 sys_catalog.cc:564] T 00000000000000000000000000000000 P b6886ade627940d896a39d91e0348019 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 06:59:52.344488 13234 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:59:52.333235 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.124:44467 (local address 127.0.0.1:34456)
0812 06:59:52.333360 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0812 06:59:52.333375 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 06:59:52.333456 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:59:52.333839 (+ 383us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:59:52.333842 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:59:52.334081 (+ 239us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:59:52.334173 (+ 92us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:59:52.334180 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:59:52.335839 (+ 1659us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:59:52.335843 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:59:52.336686 (+ 843us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:59:52.336696 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:59:52.337526 (+ 830us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:59:52.337529 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:59:52.337618 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:59:52.338120 (+ 502us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 06:59:52.338136 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:59:52.340128 (+ 1992us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 06:59:52.342367 (+ 2239us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:59:52.342370 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:59:52.342373 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:59:52.342642 (+ 269us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:59:52.343011 (+ 369us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:59:52.343014 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:59:52.343026 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:59:52.343119 (+ 93us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:59:52.343882 (+ 763us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 06:59:52.343885 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 06:59:52.344001 (+ 116us) client_negotiation.cc:769] Sending connection context
0812 06:59:52.344135 (+ 134us) client_negotiation.cc:240] Negotiation successful
0812 06:59:52.344301 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":40}
I20260812 06:59:52.345290 13399 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:59:52.333306 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34456 (local address 127.19.52.124:44467)
0812 06:59:52.333421 (+ 115us) server_negotiation.cc:206] Beginning negotiation
0812 06:59:52.333427 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 06:59:52.333439 (+ 12us) server_negotiation.cc:407] Connection header received
0812 06:59:52.333585 (+ 146us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:59:52.333591 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:59:52.333637 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:59:52.333711 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:59:52.334306 (+ 595us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:59:52.335678 (+ 1372us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:59:52.336819 (+ 1141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:59:52.337191 (+ 372us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:59:52.337291 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:59:52.340292 (+ 3001us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:59:52.340295 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:59:52.340298 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:59:52.340324 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:59:52.342245 (+ 1921us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:59:52.342751 (+ 506us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:59:52.342755 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:59:52.342756 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:59:52.342895 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:59:52.343232 (+ 337us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:59:52.343237 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:59:52.343238 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:59:52.343725 (+ 487us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:59:52.343828 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0812 06:59:52.345001 (+ 1173us) server_negotiation.cc:299] Negotiation successful
0812 06:59:52.345140 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
Aug 12 06:59:52 dist-test-slave-65mx krb5kdc[13135](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517991, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.19.52.125@KRBTEST.COM
I20260812 06:59:52.348937 13230 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 67fab72295ac409dabb8e3e247d084f1: Bootstrap starting.
W20260812 06:59:52.349093 13424 catalog_manager.cc:1558] T 00000000000000000000000000000000 P b6886ade627940d896a39d91e0348019: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 06:59:52.349171 13424 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 06:59:52.350627 13408 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:59:52.339553 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56038 (local address 127.19.52.125:46687)
0812 06:59:52.339887 (+ 334us) server_negotiation.cc:206] Beginning negotiation
0812 06:59:52.339897 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0812 06:59:52.339909 (+ 12us) server_negotiation.cc:407] Connection header received
0812 06:59:52.339942 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:59:52.339944 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:59:52.339978 (+ 34us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:59:52.340080 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:59:52.340820 (+ 740us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:59:52.341581 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:59:52.342547 (+ 966us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:59:52.342908 (+ 361us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:59:52.343361 (+ 453us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:59:52.345920 (+ 2559us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:59:52.345923 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:59:52.345926 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:59:52.345951 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:59:52.348074 (+ 2123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:59:52.348639 (+ 565us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:59:52.348642 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:59:52.348644 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:59:52.348747 (+ 103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:59:52.349061 (+ 314us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:59:52.349065 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:59:52.349067 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:59:52.349322 (+ 255us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:59:52.350207 (+ 885us) server_negotiation.cc:1035] Waiting for connection context
0812 06:59:52.350289 (+ 82us) server_negotiation.cc:299] Negotiation successful
0812 06:59:52.350420 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":284,"thread_start_us":134,"threads_started":1}
I20260812 06:59:52.351423 13230 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 67fab72295ac409dabb8e3e247d084f1: Neither blocks nor log segments found. Creating new log.
I20260812 06:59:52.352277 13230 log.cc:824] T 00000000000000000000000000000000 P 67fab72295ac409dabb8e3e247d084f1: Log is configured to *not* fsync() on all Append() calls
I20260812 06:59:52.354897 13230 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 67fab72295ac409dabb8e3e247d084f1: No bootstrap required, opened a new log
Aug 12 06:59:52 dist-test-slave-65mx krb5kdc[13135](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517992, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.52.124@KRBTEST.COM for kudu/127.19.52.125@KRBTEST.COM
I20260812 06:59:52.352918 13238 master_runner.cc:186] Error getting master registration for 127.19.52.126:34591: OK, instance_id { permanent_uuid: "67fab72295ac409dabb8e3e247d084f1" instance_seqno: 1786517991798123 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260812 06:59:52.356496 13238 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260812 06:59:52.357754 19665 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 06:59:52.359205 13199 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "67fab72295ac409dabb8e3e247d084f1"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47944
I20260812 06:59:52.360157 13230 raft_consensus.cc:348] T 00000000000000000000000000000000 P 67fab72295ac409dabb8e3e247d084f1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67fab72295ac409dabb8e3e247d084f1" member_type: VOTER last_known_addr { host: "127.19.52.126" port: 34591 } } peers { permanent_uuid: "b6886ade627940d896a39d91e0348019" member_type: VOTER last_known_addr { host: "127.19.52.125" port: 46687 } } peers { permanent_uuid: "c3e9882221ea426292aa99f5a9053215" member_type: VOTER last_known_addr { host: "127.19.52.124" port: 44467 } }
I20260812 06:59:52.360440 13230 raft_consensus.cc:729] T 00000000000000000000000000000000 P 67fab72295ac409dabb8e3e247d084f1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 67fab72295ac409dabb8e3e247d084f1, State: Initialized, Role: FOLLOWER
I20260812 06:59:52.360690 13317 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:59:52.339091 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56026 (local address 127.19.52.125:46687)
0812 06:59:52.339257 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0812 06:59:52.339263 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 06:59:52.341482 (+ 2219us) server_negotiation.cc:407] Connection header received
0812 06:59:52.342497 (+ 1015us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:59:52.342500 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:59:52.342543 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:59:52.342591 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:59:52.343218 (+ 627us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:59:52.343953 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:59:52.345281 (+ 1328us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:59:52.345595 (+ 314us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:59:52.345676 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:59:52.356068 (+ 10392us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:59:52.356073 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:59:52.356076 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:59:52.356107 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:59:52.358284 (+ 2177us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:59:52.358965 (+ 681us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:59:52.358970 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:59:52.358972 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:59:52.359068 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:59:52.359587 (+ 519us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:59:52.359591 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:59:52.359593 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:59:52.359858 (+ 265us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:59:52.359971 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0812 06:59:52.360333 (+ 362us) server_negotiation.cc:299] Negotiation successful
0812 06:59:52.360473 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":3,"server-negotiator.queue_time_us":60}
I20260812 06:59:52.360867 13230 consensus_queue.cc:260] T 00000000000000000000000000000000 P 67fab72295ac409dabb8e3e247d084f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67fab72295ac409dabb8e3e247d084f1" member_type: VOTER last_known_addr { host: "127.19.52.126" port: 34591 } } peers { permanent_uuid: "b6886ade627940d896a39d91e0348019" member_type: VOTER last_known_addr { host: "127.19.52.125" port: 46687 } } peers { permanent_uuid: "c3e9882221ea426292aa99f5a9053215" member_type: VOTER last_known_addr { host: "127.19.52.124" port: 44467 } }
I20260812 06:59:52.361310 13403 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:59:52.338866 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.125:46687 (local address 127.0.0.1:56026)
0812 06:59:52.341415 (+ 2549us) negotiation.cc:107] Waiting for socket to connect
0812 06:59:52.341428 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 06:59:52.342389 (+ 961us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:59:52.342727 (+ 338us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:59:52.342729 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:59:52.342940 (+ 211us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:59:52.343073 (+ 133us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:59:52.343083 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:59:52.344121 (+ 1038us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:59:52.344123 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:59:52.345133 (+ 1010us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:59:52.345148 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:59:52.353227 (+ 8079us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:59:52.353231 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:59:52.353369 (+ 138us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:59:52.354020 (+ 651us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 06:59:52.354043 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:59:52.355864 (+ 1821us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 06:59:52.358526 (+ 2662us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:59:52.358530 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:59:52.358533 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:59:52.358832 (+ 299us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:59:52.359197 (+ 365us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:59:52.359201 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:59:52.359205 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:59:52.359305 (+ 100us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:59:52.359992 (+ 687us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 06:59:52.359996 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 06:59:52.360118 (+ 122us) client_negotiation.cc:769] Sending connection context
0812 06:59:52.360251 (+ 133us) client_negotiation.cc:240] Negotiation successful
0812 06:59:52.360622 (+ 371us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2448}
I20260812 06:59:52.361670 13427 sys_catalog.cc:455] T 00000000000000000000000000000000 P 67fab72295ac409dabb8e3e247d084f1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67fab72295ac409dabb8e3e247d084f1" member_type: VOTER last_known_addr { host: "127.19.52.126" port: 34591 } } peers { permanent_uuid: "b6886ade627940d896a39d91e0348019" member_type: VOTER last_known_addr { host: "127.19.52.125" port: 46687 } } peers { permanent_uuid: "c3e9882221ea426292aa99f5a9053215" member_type: VOTER last_known_addr { host: "127.19.52.124" port: 44467 } } }
I20260812 06:59:52.361930 13427 sys_catalog.cc:458] T 00000000000000000000000000000000 P 67fab72295ac409dabb8e3e247d084f1 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 06:59:52.362176 13398 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.19.52.124" port: 44467 } has no permanent_uuid. Determining permanent_uuid...
I20260812 06:59:52.363433 13230 sys_catalog.cc:564] T 00000000000000000000000000000000 P 67fab72295ac409dabb8e3e247d084f1 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 06:59:52.363548 13199 raft_consensus.cc:482] T 00000000000000000000000000000000 P 67fab72295ac409dabb8e3e247d084f1 [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260812 06:59:52.363690 13199 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 67fab72295ac409dabb8e3e247d084f1 [term 0 FOLLOWER]: Advancing to term 1
I20260812 06:59:52.365808 13199 raft_consensus.cc:504] T 00000000000000000000000000000000 P 67fab72295ac409dabb8e3e247d084f1 [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67fab72295ac409dabb8e3e247d084f1" member_type: VOTER last_known_addr { host: "127.19.52.126" port: 34591 } } peers { permanent_uuid: "b6886ade627940d896a39d91e0348019" member_type: VOTER last_known_addr { host: "127.19.52.125" port: 46687 } } peers { permanent_uuid: "c3e9882221ea426292aa99f5a9053215" member_type: VOTER last_known_addr { host: "127.19.52.124" port: 44467 } }
I20260812 06:59:52.367205 13199 leader_election.cc:288] T 00000000000000000000000000000000 P 67fab72295ac409dabb8e3e247d084f1 [CANDIDATE]: Term 1 election: Requested vote from peers b6886ade627940d896a39d91e0348019 (127.19.52.125:46687), c3e9882221ea426292aa99f5a9053215 (127.19.52.124:44467)
I20260812 06:59:52.368376 13285 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "67fab72295ac409dabb8e3e247d084f1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "b6886ade627940d896a39d91e0348019"
I20260812 06:59:52.368836 13285 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b6886ade627940d896a39d91e0348019 [term 0 FOLLOWER]: Advancing to term 1
I20260812 06:59:52.370280 13285 raft_consensus.cc:2454] T 00000000000000000000000000000000 P b6886ade627940d896a39d91e0348019 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 67fab72295ac409dabb8e3e247d084f1 in term 1.
I20260812 06:59:52.370898 13172 leader_election.cc:302] T 00000000000000000000000000000000 P 67fab72295ac409dabb8e3e247d084f1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 67fab72295ac409dabb8e3e247d084f1, b6886ade627940d896a39d91e0348019; no voters:
I20260812 06:59:52.371236 13427 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 67fab72295ac409dabb8e3e247d084f1 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 06:59:52.371462 13427 raft_consensus.cc:686] T 00000000000000000000000000000000 P 67fab72295ac409dabb8e3e247d084f1 [term 1 LEADER]: Becoming Leader. State: Replica: 67fab72295ac409dabb8e3e247d084f1, State: Running, Role: LEADER
I20260812 06:59:52.371826 13427 consensus_queue.cc:237] T 00000000000000000000000000000000 P 67fab72295ac409dabb8e3e247d084f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67fab72295ac409dabb8e3e247d084f1" member_type: VOTER last_known_addr { host: "127.19.52.126" port: 34591 } } peers { permanent_uuid: "b6886ade627940d896a39d91e0348019" member_type: VOTER last_known_addr { host: "127.19.52.125" port: 46687 } } peers { permanent_uuid: "c3e9882221ea426292aa99f5a9053215" member_type: VOTER last_known_addr { host: "127.19.52.124" port: 44467 } }
W20260812 06:59:52.372036 13439 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 67fab72295ac409dabb8e3e247d084f1: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 06:59:52.372138 13439 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 06:59:52.373853 13427 sys_catalog.cc:455] T 00000000000000000000000000000000 P 67fab72295ac409dabb8e3e247d084f1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 67fab72295ac409dabb8e3e247d084f1. Latest consensus state: current_term: 1 leader_uuid: "67fab72295ac409dabb8e3e247d084f1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67fab72295ac409dabb8e3e247d084f1" member_type: VOTER last_known_addr { host: "127.19.52.126" port: 34591 } } peers { permanent_uuid: "b6886ade627940d896a39d91e0348019" member_type: VOTER last_known_addr { host: "127.19.52.125" port: 46687 } } peers { permanent_uuid: "c3e9882221ea426292aa99f5a9053215" member_type: VOTER last_known_addr { host: "127.19.52.124" port: 44467 } } }
I20260812 06:59:52.373993 13427 sys_catalog.cc:458] T 00000000000000000000000000000000 P 67fab72295ac409dabb8e3e247d084f1 [sys.catalog]: This master's current role is: LEADER
I20260812 06:59:52.380332 13445 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 06:59:52.381803 13445 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 06:59:52.382495 13153 master_runner.cc:186] Error getting master registration for 127.19.52.124:44467: OK, instance_id { permanent_uuid: "c3e9882221ea426292aa99f5a9053215" instance_seqno: 1786517992306885 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260812 06:59:52.386140 13153 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
Aug 12 06:59:52 dist-test-slave-65mx krb5kdc[13135](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517992, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.52.124@KRBTEST.COM for kudu/127.19.52.124@KRBTEST.COM
I20260812 06:59:52.392226 13403 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:59:52.362722 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.52.124:44467 (local address 127.0.0.1:34472)
0812 06:59:52.362841 (+ 119us) negotiation.cc:107] Waiting for socket to connect
0812 06:59:52.362859 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0812 06:59:52.362944 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:59:52.363230 (+ 286us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:59:52.363233 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:59:52.363426 (+ 193us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:59:52.363556 (+ 130us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:59:52.363564 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:59:52.365754 (+ 2190us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:59:52.365756 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:59:52.366503 (+ 747us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:59:52.366515 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:59:52.379016 (+ 12501us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:59:52.379021 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:59:52.379161 (+ 140us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:59:52.386621 (+ 7460us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 06:59:52.386638 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:59:52.388361 (+ 1723us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 06:59:52.390617 (+ 2256us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:59:52.390620 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:59:52.390622 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:59:52.390880 (+ 258us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:59:52.391187 (+ 307us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:59:52.391189 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:59:52.391191 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:59:52.391273 (+ 82us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:59:52.391718 (+ 445us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 06:59:52.391721 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 06:59:52.391809 (+ 88us) client_negotiation.cc:769] Sending connection context
0812 06:59:52.391899 (+ 90us) client_negotiation.cc:240] Negotiation successful
0812 06:59:52.392046 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":33}
I20260812 06:59:52.392714 13399 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:59:52.362779 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34472 (local address 127.19.52.124:44467)
0812 06:59:52.362893 (+ 114us) server_negotiation.cc:206] Beginning negotiation
0812 06:59:52.362897 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 06:59:52.362912 (+ 15us) server_negotiation.cc:407] Connection header received
0812 06:59:52.363044 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:59:52.363047 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:59:52.363107 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:59:52.363190 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:59:52.363687 (+ 497us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:59:52.364588 (+ 901us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:59:52.366642 (+ 2054us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:59:52.366989 (+ 347us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:59:52.367100 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:59:52.388515 (+ 21415us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:59:52.388519 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:59:52.388521 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:59:52.388575 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:59:52.390478 (+ 1903us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:59:52.390993 (+ 515us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:59:52.390996 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:59:52.390998 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:59:52.391093 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:59:52.391364 (+ 271us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:59:52.391367 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:59:52.391368 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:59:52.391585 (+ 217us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:59:52.391673 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0812 06:59:52.392454 (+ 781us) server_negotiation.cc:299] Negotiation successful
0812 06:59:52.392592 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260812 06:59:52.396595 13398 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c3e9882221ea426292aa99f5a9053215: Bootstrap starting.
I20260812 06:59:52.399612 13398 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c3e9882221ea426292aa99f5a9053215: Neither blocks nor log segments found. Creating new log.
I20260812 06:59:52.401015 13398 log.cc:824] T 00000000000000000000000000000000 P c3e9882221ea426292aa99f5a9053215: Log is configured to *not* fsync() on all Append() calls
I20260812 06:59:52.405308 13398 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c3e9882221ea426292aa99f5a9053215: No bootstrap required, opened a new log
I20260812 06:59:52.409090 13402 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:59:52.377342 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34482 (local address 127.19.52.124:44467)
0812 06:59:52.377473 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0812 06:59:52.377487 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0812 06:59:52.384267 (+ 6780us) server_negotiation.cc:407] Connection header received
0812 06:59:52.384434 (+ 167us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:59:52.384437 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:59:52.384491 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:59:52.384564 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:59:52.385678 (+ 1114us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:59:52.386700 (+ 1022us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:59:52.388141 (+ 1441us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:59:52.388538 (+ 397us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:59:52.388657 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:59:52.391585 (+ 2928us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:59:52.391589 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:59:52.391592 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:59:52.391620 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:59:52.393755 (+ 2135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:59:52.407523 (+ 13768us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:59:52.407528 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:59:52.407530 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:59:52.407646 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:59:52.407980 (+ 334us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:59:52.407983 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:59:52.407985 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:59:52.408221 (+ 236us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:59:52.408342 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0812 06:59:52.408781 (+ 439us) server_negotiation.cc:299] Negotiation successful
0812 06:59:52.408921 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260812 06:59:52.411655 13405 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:59:52.376976 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47946 (local address 127.19.52.126:34591)
0812 06:59:52.377107 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0812 06:59:52.377112 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 06:59:52.386226 (+ 9114us) server_negotiation.cc:407] Connection header received
0812 06:59:52.386274 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:59:52.386277 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:59:52.386339 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:59:52.386428 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:59:52.396449 (+ 10021us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:59:52.397326 (+ 877us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:59:52.399164 (+ 1838us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:59:52.399556 (+ 392us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:59:52.399640 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:59:52.403003 (+ 3363us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:59:52.403007 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:59:52.403010 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:59:52.403043 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:59:52.404867 (+ 1824us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:59:52.410046 (+ 5179us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:59:52.410050 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:59:52.410052 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:59:52.410185 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:59:52.410527 (+ 342us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:59:52.410532 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:59:52.410534 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:59:52.410788 (+ 254us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:59:52.410932 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0812 06:59:52.411345 (+ 413us) server_negotiation.cc:299] Negotiation successful
0812 06:59:52.411487 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260812 06:59:52.412911 13285 raft_consensus.cc:1261] T 00000000000000000000000000000000 P b6886ade627940d896a39d91e0348019 [term 1 FOLLOWER]: Refusing update from remote peer 67fab72295ac409dabb8e3e247d084f1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 06:59:52.413975 13442 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 67fab72295ac409dabb8e3e247d084f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b6886ade627940d896a39d91e0348019" member_type: VOTER last_known_addr { host: "127.19.52.125" port: 46687 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 06:59:52.414476 13398 raft_consensus.cc:348] T 00000000000000000000000000000000 P c3e9882221ea426292aa99f5a9053215 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67fab72295ac409dabb8e3e247d084f1" member_type: VOTER last_known_addr { host: "127.19.52.126" port: 34591 } } peers { permanent_uuid: "b6886ade627940d896a39d91e0348019" member_type: VOTER last_known_addr { host: "127.19.52.125" port: 46687 } } peers { permanent_uuid: "c3e9882221ea426292aa99f5a9053215" member_type: VOTER last_known_addr { host: "127.19.52.124" port: 44467 } }
I20260812 06:59:52.414790 13398 raft_consensus.cc:729] T 00000000000000000000000000000000 P c3e9882221ea426292aa99f5a9053215 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c3e9882221ea426292aa99f5a9053215, State: Initialized, Role: FOLLOWER
I20260812 06:59:52.415210 13398 consensus_queue.cc:260] T 00000000000000000000000000000000 P c3e9882221ea426292aa99f5a9053215 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67fab72295ac409dabb8e3e247d084f1" member_type: VOTER last_known_addr { host: "127.19.52.126" port: 34591 } } peers { permanent_uuid: "b6886ade627940d896a39d91e0348019" member_type: VOTER last_known_addr { host: "127.19.52.125" port: 46687 } } peers { permanent_uuid: "c3e9882221ea426292aa99f5a9053215" member_type: VOTER last_known_addr { host: "127.19.52.124" port: 44467 } }
I20260812 06:59:52.416796 13398 sys_catalog.cc:564] T 00000000000000000000000000000000 P c3e9882221ea426292aa99f5a9053215 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 06:59:52.417062 13317 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:59:52.389283 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56040 (local address 127.19.52.125:46687)
0812 06:59:52.389421 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0812 06:59:52.389426 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 06:59:52.400959 (+ 11533us) server_negotiation.cc:407] Connection header received
0812 06:59:52.401098 (+ 139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:59:52.401102 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:59:52.401160 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:59:52.401247 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:59:52.405406 (+ 4159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:59:52.406254 (+ 848us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:59:52.407502 (+ 1248us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:59:52.407895 (+ 393us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:59:52.407995 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:59:52.411453 (+ 3458us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:59:52.411458 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:59:52.411461 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:59:52.411495 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:59:52.414602 (+ 3107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:59:52.415152 (+ 550us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:59:52.415156 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:59:52.415158 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:59:52.415259 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:59:52.415611 (+ 352us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:59:52.415615 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:59:52.415616 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:59:52.416108 (+ 492us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:59:52.416228 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0812 06:59:52.416630 (+ 402us) server_negotiation.cc:299] Negotiation successful
0812 06:59:52.416896 (+ 266us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260812 06:59:52.418603 13369 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c3e9882221ea426292aa99f5a9053215 [term 0 FOLLOWER]: Advancing to term 1
I20260812 06:59:52.418900 13409 sys_catalog.cc:455] T 00000000000000000000000000000000 P b6886ade627940d896a39d91e0348019 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 67fab72295ac409dabb8e3e247d084f1. Latest consensus state: current_term: 1 leader_uuid: "67fab72295ac409dabb8e3e247d084f1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67fab72295ac409dabb8e3e247d084f1" member_type: VOTER last_known_addr { host: "127.19.52.126" port: 34591 } } peers { permanent_uuid: "b6886ade627940d896a39d91e0348019" member_type: VOTER last_known_addr { host: "127.19.52.125" port: 46687 } } peers { permanent_uuid: "c3e9882221ea426292aa99f5a9053215" member_type: VOTER last_known_addr { host: "127.19.52.124" port: 44467 } } }
I20260812 06:59:52.419045 13409 sys_catalog.cc:458] T 00000000000000000000000000000000 P b6886ade627940d896a39d91e0348019 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 06:59:52.420148 13369 raft_consensus.cc:1261] T 00000000000000000000000000000000 P c3e9882221ea426292aa99f5a9053215 [term 1 FOLLOWER]: Refusing update from remote peer 67fab72295ac409dabb8e3e247d084f1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 06:59:52.421211 13370 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "67fab72295ac409dabb8e3e247d084f1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "c3e9882221ea426292aa99f5a9053215"
I20260812 06:59:52.423087 13442 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 67fab72295ac409dabb8e3e247d084f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c3e9882221ea426292aa99f5a9053215" member_type: VOTER last_known_addr { host: "127.19.52.124" port: 44467 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 06:59:52.422822 13370 raft_consensus.cc:2454] T 00000000000000000000000000000000 P c3e9882221ea426292aa99f5a9053215 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 67fab72295ac409dabb8e3e247d084f1 in term 1.
I20260812 06:59:52.432145 13442 sys_catalog.cc:455] T 00000000000000000000000000000000 P 67fab72295ac409dabb8e3e247d084f1 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "67fab72295ac409dabb8e3e247d084f1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67fab72295ac409dabb8e3e247d084f1" member_type: VOTER last_known_addr { host: "127.19.52.126" port: 34591 } } peers { permanent_uuid: "b6886ade627940d896a39d91e0348019" member_type: VOTER last_known_addr { host: "127.19.52.125" port: 46687 } } peers { permanent_uuid: "c3e9882221ea426292aa99f5a9053215" member_type: VOTER last_known_addr { host: "127.19.52.124" port: 44467 } } }
I20260812 06:59:52.432322 13442 sys_catalog.cc:458] T 00000000000000000000000000000000 P 67fab72295ac409dabb8e3e247d084f1 [sys.catalog]: This master's current role is: LEADER
I20260812 06:59:52.434000 13459 mvcc.cc:205] Tried to move back new op lower bound from 7317577696904732672 to 7317577696759078912. Current Snapshot: MvccSnapshot[applied={T|T < 7317577696904732672}]
I20260812 06:59:52.434077 13473 sys_catalog.cc:455] T 00000000000000000000000000000000 P c3e9882221ea426292aa99f5a9053215 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 67fab72295ac409dabb8e3e247d084f1. Latest consensus state: current_term: 1 leader_uuid: "67fab72295ac409dabb8e3e247d084f1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67fab72295ac409dabb8e3e247d084f1" member_type: VOTER last_known_addr { host: "127.19.52.126" port: 34591 } } peers { permanent_uuid: "b6886ade627940d896a39d91e0348019" member_type: VOTER last_known_addr { host: "127.19.52.125" port: 46687 } } peers { permanent_uuid: "c3e9882221ea426292aa99f5a9053215" member_type: VOTER last_known_addr { host: "127.19.52.124" port: 44467 } } }
I20260812 06:59:52.434213 13473 sys_catalog.cc:458] T 00000000000000000000000000000000 P c3e9882221ea426292aa99f5a9053215 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 06:59:52.434500 13409 sys_catalog.cc:455] T 00000000000000000000000000000000 P b6886ade627940d896a39d91e0348019 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "67fab72295ac409dabb8e3e247d084f1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67fab72295ac409dabb8e3e247d084f1" member_type: VOTER last_known_addr { host: "127.19.52.126" port: 34591 } } peers { permanent_uuid: "b6886ade627940d896a39d91e0348019" member_type: VOTER last_known_addr { host: "127.19.52.125" port: 46687 } } peers { permanent_uuid: "c3e9882221ea426292aa99f5a9053215" member_type: VOTER last_known_addr { host: "127.19.52.124" port: 44467 } } }
I20260812 06:59:52.434619 13409 sys_catalog.cc:458] T 00000000000000000000000000000000 P b6886ade627940d896a39d91e0348019 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 06:59:52.437381 13458 sys_catalog.cc:455] T 00000000000000000000000000000000 P c3e9882221ea426292aa99f5a9053215 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "67fab72295ac409dabb8e3e247d084f1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67fab72295ac409dabb8e3e247d084f1" member_type: VOTER last_known_addr { host: "127.19.52.126" port: 34591 } } peers { permanent_uuid: "b6886ade627940d896a39d91e0348019" member_type: VOTER last_known_addr { host: "127.19.52.125" port: 46687 } } peers { permanent_uuid: "c3e9882221ea426292aa99f5a9053215" member_type: VOTER last_known_addr { host: "127.19.52.124" port: 44467 } } }
I20260812 06:59:52.437620 13458 sys_catalog.cc:458] T 00000000000000000000000000000000 P c3e9882221ea426292aa99f5a9053215 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 06:59:52.438403 13427 sys_catalog.cc:455] T 00000000000000000000000000000000 P 67fab72295ac409dabb8e3e247d084f1 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "67fab72295ac409dabb8e3e247d084f1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67fab72295ac409dabb8e3e247d084f1" member_type: VOTER last_known_addr { host: "127.19.52.126" port: 34591 } } peers { permanent_uuid: "b6886ade627940d896a39d91e0348019" member_type: VOTER last_known_addr { host: "127.19.52.125" port: 46687 } } peers { permanent_uuid: "c3e9882221ea426292aa99f5a9053215" member_type: VOTER last_known_addr { host: "127.19.52.124" port: 44467 } } }
I20260812 06:59:52.438657 13427 sys_catalog.cc:458] T 00000000000000000000000000000000 P 67fab72295ac409dabb8e3e247d084f1 [sys.catalog]: This master's current role is: LEADER
I20260812 06:59:52.439375 13455 mvcc.cc:205] Tried to move back new op lower bound from 7317577696904732672 to 7317577696759078912. Current Snapshot: MvccSnapshot[applied={T|T < 7317577696904732672}]
I20260812 06:59:52.443667 13458 sys_catalog.cc:455] T 00000000000000000000000000000000 P c3e9882221ea426292aa99f5a9053215 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "67fab72295ac409dabb8e3e247d084f1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67fab72295ac409dabb8e3e247d084f1" member_type: VOTER last_known_addr { host: "127.19.52.126" port: 34591 } } peers { permanent_uuid: "b6886ade627940d896a39d91e0348019" member_type: VOTER last_known_addr { host: "127.19.52.125" port: 46687 } } peers { permanent_uuid: "c3e9882221ea426292aa99f5a9053215" member_type: VOTER last_known_addr { host: "127.19.52.124" port: 44467 } } }
I20260812 06:59:52.443799 13458 sys_catalog.cc:458] T 00000000000000000000000000000000 P c3e9882221ea426292aa99f5a9053215 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 06:59:52.444326 13445 catalog_manager.cc:1347] Generated new cluster ID: 289e1397a1334526af02f252ba1be680
I20260812 06:59:52.444402 13445 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
W20260812 06:59:52.445014 13472 catalog_manager.cc:1558] T 00000000000000000000000000000000 P c3e9882221ea426292aa99f5a9053215: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 06:59:52.445101 13472 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 06:59:52.466162 13445 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 06:59:52.467665 13445 catalog_manager.cc:1504] Loading token signing keys...
I20260812 06:59:52.485911 13445 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 67fab72295ac409dabb8e3e247d084f1: Generated new TSK 0
I20260812 06:59:52.486928 13445 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260812 06:59:52.528045 13405 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 06:59:52.525073 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47948 (local address 127.19.52.126:34591)
0812 06:59:52.525230 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0812 06:59:52.525237 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 06:59:52.526567 (+ 1330us) server_negotiation.cc:407] Connection header received
0812 06:59:52.527033 (+ 466us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:59:52.527037 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:59:52.527092 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:59:52.527161 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:59:52.527895 (+ 734us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:47948: BlockingRecv error: recv got EOF from 127.0.0.1:47948 (error 108)
Metrics: {"server-negotiator.queue_time_us":55}
W20260812 06:59:52.529335 13402 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 06:59:52.525889 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34484 (local address 127.19.52.124:44467)
0812 06:59:52.526005 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0812 06:59:52.526009 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 06:59:52.526947 (+ 938us) server_negotiation.cc:407] Connection header received
0812 06:59:52.528437 (+ 1490us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:59:52.528440 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:59:52.528494 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:59:52.528593 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:59:52.529217 (+ 624us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34484: BlockingRecv error: recv got EOF from 127.0.0.1:34484 (error 108)
Metrics: {"server-negotiator.queue_time_us":47}
W20260812 06:59:52.531620 13317 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 06:59:52.530216 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56042 (local address 127.19.52.125:46687)
0812 06:59:52.530346 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0812 06:59:52.530351 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 06:59:52.530670 (+ 319us) server_negotiation.cc:407] Connection header received
0812 06:59:52.530850 (+ 180us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:59:52.530854 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:59:52.530904 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:59:52.530958 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:59:52.531499 (+ 541us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:56042: BlockingRecv error: recv got EOF from 127.0.0.1:56042 (error 108)
Metrics: {"server-negotiator.queue_time_us":56}
W20260812 06:59:52.536257 13317 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 06:59:52.534617 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56052 (local address 127.19.52.125:46687)
0812 06:59:52.534724 (+ 107us) server_negotiation.cc:206] Beginning negotiation
0812 06:59:52.534729 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 06:59:52.535254 (+ 525us) server_negotiation.cc:407] Connection header received
0812 06:59:52.535467 (+ 213us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:59:52.535470 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:59:52.535556 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:59:52.535611 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:59:52.536138 (+ 527us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:56052: BlockingRecv error: recv got EOF from 127.0.0.1:56052 (error 108)
Metrics: {"server-negotiator.queue_time_us":44}
W20260812 06:59:52.541713 13317 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 06:59:52.540069 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56064 (local address 127.19.52.125:46687)
0812 06:59:52.540190 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0812 06:59:52.540195 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 06:59:52.540717 (+ 522us) server_negotiation.cc:407] Connection header received
0812 06:59:52.540924 (+ 207us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:59:52.540927 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:59:52.540992 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:59:52.541051 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:59:52.541606 (+ 555us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:56064: BlockingRecv error: recv got EOF from 127.0.0.1:56064 (error 108)
Metrics: {"server-negotiator.queue_time_us":43}
W20260812 06:59:52.549199 13317 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 06:59:52.547414 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56076 (local address 127.19.52.125:46687)
0812 06:59:52.547530 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0812 06:59:52.547535 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 06:59:52.548070 (+ 535us) server_negotiation.cc:407] Connection header received
0812 06:59:52.548318 (+ 248us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:59:52.548322 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:59:52.548380 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:59:52.548442 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:59:52.549057 (+ 615us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:56076: BlockingRecv error: recv got EOF from 127.0.0.1:56076 (error 108)
Metrics: {"server-negotiator.queue_time_us":43}
W20260812 06:59:52.558161 13317 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 06:59:52.556134 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56080 (local address 127.19.52.125:46687)
0812 06:59:52.556271 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0812 06:59:52.556276 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 06:59:52.556921 (+ 645us) server_negotiation.cc:407] Connection header received
0812 06:59:52.557216 (+ 295us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:59:52.557220 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:59:52.557285 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:59:52.557349 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:59:52.558027 (+ 678us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:56080: BlockingRecv error: recv got EOF from 127.0.0.1:56080 (error 108)
Metrics: {"server-negotiator.queue_time_us":57}
W20260812 06:59:52.571364 13317 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 06:59:52.569343 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56088 (local address 127.19.52.125:46687)
0812 06:59:52.569476 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0812 06:59:52.569481 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 06:59:52.570176 (+ 695us) server_negotiation.cc:407] Connection header received
0812 06:59:52.570422 (+ 246us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:59:52.570427 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:59:52.570502 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:59:52.570578 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:59:52.571237 (+ 659us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:56088: BlockingRecv error: recv got EOF from 127.0.0.1:56088 (error 108)
Metrics: {"server-negotiator.queue_time_us":52}
W20260812 06:59:52.591516 13317 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 06:59:52.590123 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56096 (local address 127.19.52.125:46687)
0812 06:59:52.590258 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0812 06:59:52.590264 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 06:59:52.590526 (+ 262us) server_negotiation.cc:407] Connection header received
0812 06:59:52.590699 (+ 173us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:59:52.590703 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:59:52.590762 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:59:52.590858 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:59:52.591361 (+ 503us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:56096: BlockingRecv error: recv got EOF from 127.0.0.1:56096 (error 108)
Metrics: {"server-negotiator.queue_time_us":55}
W20260812 06:59:52.627104 13317 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 06:59:52.625893 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56108 (local address 127.19.52.125:46687)
0812 06:59:52.626044 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0812 06:59:52.626049 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 06:59:52.626175 (+ 126us) server_negotiation.cc:407] Connection header received
0812 06:59:52.626316 (+ 141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:59:52.626319 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:59:52.626391 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:59:52.626483 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:59:52.626973 (+ 490us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:56108: BlockingRecv error: recv got EOF from 127.0.0.1:56108 (error 108)
Metrics: {"server-negotiator.queue_time_us":68}
W20260812 06:59:52.694804 13317 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 06:59:52.693483 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56122 (local address 127.19.52.125:46687)
0812 06:59:52.693682 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0812 06:59:52.693686 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 06:59:52.693791 (+ 105us) server_negotiation.cc:407] Connection header received
0812 06:59:52.693924 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:59:52.693926 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:59:52.693980 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:59:52.694055 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:59:52.694677 (+ 622us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:56122: BlockingRecv error: recv got EOF from 127.0.0.1:56122 (error 108)
Metrics: {"server-negotiator.queue_time_us":94}
W20260812 06:59:52.826224 13317 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 06:59:52.825021 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56124 (local address 127.19.52.125:46687)
0812 06:59:52.825223 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0812 06:59:52.825227 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 06:59:52.825312 (+ 85us) server_negotiation.cc:407] Connection header received
0812 06:59:52.825437 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:59:52.825439 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:59:52.825494 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:59:52.825568 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:59:52.826110 (+ 542us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:56124: BlockingRecv error: recv got EOF from 127.0.0.1:56124 (error 108)
Metrics: {"server-negotiator.queue_time_us":98}
W20260812 06:59:53.085755 13317 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 06:59:53.084507 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56128 (local address 127.19.52.125:46687)
0812 06:59:53.084707 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0812 06:59:53.084714 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 06:59:53.084732 (+ 18us) server_negotiation.cc:407] Connection header received
0812 06:59:53.084922 (+ 190us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:59:53.084926 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:59:53.084992 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:59:53.085091 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:59:53.085616 (+ 525us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:56128: BlockingRecv error: recv got EOF from 127.0.0.1:56128 (error 108)
Metrics: {"server-negotiator.queue_time_us":101}
I20260812 06:59:53.350590 13424 catalog_manager.cc:1259] Loaded cluster ID: 289e1397a1334526af02f252ba1be680
I20260812 06:59:53.350700 13424 catalog_manager.cc:1552] T 00000000000000000000000000000000 P b6886ade627940d896a39d91e0348019: loading cluster ID for follower catalog manager: success
I20260812 06:59:53.352325 13424 catalog_manager.cc:1574] T 00000000000000000000000000000000 P b6886ade627940d896a39d91e0348019: acquiring CA information for follower catalog manager: success
I20260812 06:59:53.353461 13424 catalog_manager.cc:1602] T 00000000000000000000000000000000 P b6886ade627940d896a39d91e0348019: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260812 06:59:53.446439 13472 catalog_manager.cc:1259] Loaded cluster ID: 289e1397a1334526af02f252ba1be680
I20260812 06:59:53.446573 13472 catalog_manager.cc:1552] T 00000000000000000000000000000000 P c3e9882221ea426292aa99f5a9053215: loading cluster ID for follower catalog manager: success
I20260812 06:59:53.448299 13472 catalog_manager.cc:1574] T 00000000000000000000000000000000 P c3e9882221ea426292aa99f5a9053215: acquiring CA information for follower catalog manager: success
I20260812 06:59:53.449522 13472 catalog_manager.cc:1602] T 00000000000000000000000000000000 P c3e9882221ea426292aa99f5a9053215: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260812 06:59:53.604022 13556 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:59:53.599961 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56144 (local address 127.19.52.125:46687)
0812 06:59:53.600249 (+ 288us) server_negotiation.cc:206] Beginning negotiation
0812 06:59:53.600253 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 06:59:53.600270 (+ 17us) server_negotiation.cc:407] Connection header received
0812 06:59:53.600455 (+ 185us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:59:53.600459 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:59:53.600629 (+ 170us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:59:53.600735 (+ 106us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 06:59:53.601044 (+ 309us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:59:53.601576 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:59:53.602441 (+ 865us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:59:53.602680 (+ 239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:59:53.602769 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:59:53.603159 (+ 390us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 06:59:53.603604 (+ 445us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 06:59:53.603717 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0812 06:59:53.603835 (+ 118us) server_negotiation.cc:299] Negotiation successful
0812 06:59:53.603891 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":207,"thread_start_us":81,"threads_started":1}
I20260812 06:59:53.611645 13562 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:59:53.607279 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34492 (local address 127.19.52.124:44467)
0812 06:59:53.607592 (+ 313us) server_negotiation.cc:206] Beginning negotiation
0812 06:59:53.607598 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 06:59:53.607629 (+ 31us) server_negotiation.cc:407] Connection header received
0812 06:59:53.607721 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:59:53.607724 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:59:53.607841 (+ 117us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:59:53.607944 (+ 103us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 06:59:53.608231 (+ 287us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:59:53.608961 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:59:53.609898 (+ 937us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:59:53.610171 (+ 273us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:59:53.610266 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:59:53.610621 (+ 355us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 06:59:53.611111 (+ 490us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 06:59:53.611239 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0812 06:59:53.611313 (+ 74us) server_negotiation.cc:299] Negotiation successful
0812 06:59:53.611373 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":220,"thread_start_us":96,"threads_started":1}
I20260812 06:59:53.619758 13562 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:59:53.615790 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34498 (local address 127.19.52.124:44467)
0812 06:59:53.615927 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0812 06:59:53.615931 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 06:59:53.616163 (+ 232us) server_negotiation.cc:407] Connection header received
0812 06:59:53.616216 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:59:53.616218 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:59:53.616260 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:59:53.616343 (+ 83us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 06:59:53.616624 (+ 281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:59:53.617255 (+ 631us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:59:53.618427 (+ 1172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:59:53.618830 (+ 403us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:59:53.618954 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:59:53.619208 (+ 254us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 06:59:53.619287 (+ 79us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 06:59:53.619374 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0812 06:59:53.619505 (+ 131us) server_negotiation.cc:299] Negotiation successful
0812 06:59:53.619572 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260812 06:59:53.619992 13556 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:59:53.615436 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56148 (local address 127.19.52.125:46687)
0812 06:59:53.615639 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0812 06:59:53.615645 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 06:59:53.616007 (+ 362us) server_negotiation.cc:407] Connection header received
0812 06:59:53.616186 (+ 179us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:59:53.616191 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:59:53.616255 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:59:53.616330 (+ 75us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 06:59:53.616784 (+ 454us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:59:53.617629 (+ 845us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:59:53.618700 (+ 1071us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:59:53.619030 (+ 330us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:59:53.619145 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:59:53.619428 (+ 283us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 06:59:53.619513 (+ 85us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 06:59:53.619569 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0812 06:59:53.619811 (+ 242us) server_negotiation.cc:299] Negotiation successful
0812 06:59:53.619872 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260812 06:59:53.620843 13569 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:59:53.615343 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47962 (local address 127.19.52.126:34591)
0812 06:59:53.616177 (+ 834us) server_negotiation.cc:206] Beginning negotiation
0812 06:59:53.616182 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 06:59:53.616195 (+ 13us) server_negotiation.cc:407] Connection header received
0812 06:59:53.616295 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:59:53.616300 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:59:53.616481 (+ 181us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:59:53.616689 (+ 208us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 06:59:53.616916 (+ 227us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:59:53.617708 (+ 792us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:59:53.618899 (+ 1191us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:59:53.619241 (+ 342us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:59:53.619360 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:59:53.619806 (+ 446us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 06:59:53.620188 (+ 382us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 06:59:53.620277 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0812 06:59:53.620419 (+ 142us) server_negotiation.cc:299] Negotiation successful
0812 06:59:53.620478 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":746,"thread_start_us":68,"threads_started":1}
I20260812 06:59:53.628084 19665 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu with pid 13153
I20260812 06:59:53.638643 19665 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu with pid 13238
I20260812 06:59:53.648241 19665 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskG7Vh1a/build/debug/bin/kudu with pid 13318
2026-08-12T06:59:53Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1 (3322 ms)
[----------] 2 tests from ConnectToFollowerMasterTest (6730 ms total)
[----------] Global test environment tear-down
[==========] 56 tests from 4 test suites ran. (306227 ms total)
[ PASSED ] 55 tests.
[ FAILED ] 1 test, listed below:
[ FAILED ] SecurityITest.TestJwtMiniCluster
1 FAILED TEST